Improving safety and reducing error in endoscopy: simulation training in human factors

[Construction of a methodology for clinical evaluation of medical devices using simulation tools and illustration through three studies]

INTRODUCTION

European regulations have recently moved towards more stringent requirements for demonstrating the safety and performance of medical devices (MDs).

OBJECTIVE

To apply an innovative testing method using medical simulation to the evaluation of three medical devices at different stages of their life cycle.

METHOD

The methodology for evaluating DMs using simulation is based on seven stages: definition of the context, training, construction of a scenario to test the DM, validation of the scenario, realization of the scenario, evaluation of the scenario by the players and validation and exploitation of the results.

RESULTS

Our evaluation methodology enabled us to assess three DMs at different stages of their development: a respiratory protection device at the initial stage (prototype definition), a respiratory protection mask (prototype optimization) and bottle adapters (post-marketing).

CONCLUSION

Simulation is a valuable tool for evaluating DM. The proposed methodology enables it to be used and adapted to different contexts. It responds to the specificities of clinical evaluation of this class of products, and helps to better anticipate certain risks.

Training in basic gastrointestinal endoscopic procedures: a European Society of Gastrointestinal Endoscopy (ESGE) and European Society of Gastroenterology and Endoscopy Nurses and Associates (ESGENA) Position Statement

This ESGE Position Statement provides structured and evidence-based guidance on the essential requirements and processes involved in training in basic gastrointestinal (GI) endoscopic procedures. The document outlines definitions; competencies required, and means to their assessment and maintenance; the structure and requirements of training programs; patient safety and medicolegal issues. 1:  ESGE and ESGENA define basic endoscopic procedures as those procedures that are commonly indicated, generally accessible, and expected to be mastered (technically and cognitively) by the end of any core training program in gastrointestinal endoscopy. 2:  ESGE and ESGENA consider the following as basic endoscopic procedures: diagnostic upper and lower GI endoscopy, as well as a limited range of interventions such as: tissue acquisition via cold biopsy forceps, polypectomy for lesions ≤ 10 mm, hemostasis techniques, enteral feeding tube placement, foreign body retrieval, dilation of simple esophageal strictures, and India ink tattooing of lesion location. 3:  ESGE and ESGENA recommend that training in GI endoscopy should be subject to stringent formal requirements that ensure all ESGE key performance indicators (KPIs) are met. 4:  Training in basic endoscopic procedures is a complex process and includes the development and acquisition of cognitive, technical/motor, and integrative skills. Therefore, ESGE and ESGENA recommend the use of validated tools to track the development of skills and assess competence. 5:  ESGE and ESGENA recommend incorporating a multimodal approach to evaluating competence in basic GI endoscopic procedures, including procedural thresholds and the measurement and documentation of established ESGE KPIs. 7:  ESGE and ESGENA recommend the continuous monitoring of ESGE KPIs during GI endoscopy training to ensure the trainee's maintenance of competence. 9:  ESGE and ESGENA recommend that GI endoscopy training units fulfil the ESGE KPIs for endoscopy units and, furthermore, be capable of providing the dedicated personnel, infrastructure, and sufficient case volume required for successful training within a structured training program. 10:  ESGE and ESGENA recommend that trainers in basic GI endoscopic procedures should be endoscopists with formal educational training in the teaching of endoscopy, which allows them to successfully and safely teach trainees.

Development and validation of a simulation-based assessment tool in colonoscopy

BACKGROUND

Colonoscopy is difficult to learn. Virtual reality simulation training is helpful, but how and when novices should progress to patient-based training has yet to be established. To date, there is no assessment tool for credentialing novice endoscopists prior to clinical practice. The aim of this study was to develop such an assessment tool based on metrics provided by the simulator. The metrics used for the assessment tool should be able to discriminate between novices, intermediates, and experts and include essential checklist items for patient safety.

METHODS

The validation process was conducted based on the Standards for Educational and Psychological Testing. An expert panel decided upon three essential checklist items for patient safety based on Lawshe's method: perforation, hazardous tension to the bowel wall, and cecal intubation. A power calculation was performed. In this study, the Simbionix GI Mentor II simulator was used. Metrics with discriminatory ability were identified with variance analysis and combined to form an aggregate score. Based on this score and the essential items, pass/fail standards were set and reliability was tested.

RESULTS

Twenty-four participants (eight novices, eight intermediates, and eight expert endoscopists) performed two simulated colonoscopies. Four metrics with discriminatory ability were identified. The aggregate score ranged from 4.2 to 51.2 points. Novices had a mean score of 10.00 (SD 5.13), intermediates 24.63 (SD 7.91), and experts 30.72 (SD 11.98). The difference in score between novices and the other two groups was statistically significant (p<0.01). Although expert endoscopists had a higher score, the difference was not statistically significant (p=0.40). Reliability was good (Cronbach's alpha=0.86). A pass/fail score was defined at 17.1 points with correct completion of three essential checklist items, resulting in three experts and three intermediates failing and one novice passing the assessment.

CONCLUSION

We established a valid and reliable assessment tool with a pass/fail standard on the simulator. We suggest using the assessment after simulation-based training before commencing work-based learning.

Who will excel in advanced endoscopy? A study assessing the criteria and perceptions of experts with regard to selection of ERCP and EUS trainees

Background and study aims  Training program directors (TPDs) and experts play a crucial role in selecting ERCP/EUS trainees and determining the workforce in endoscopy. Additionally, prospective trainees should know what TPDs/experts expect from them. Nonetheless, the criteria and perceptions used in this selection have not been clarified. The aim of this study was to identify TPD/expert values/beliefs regarding personal attributes needed for selecting trainees that can excel and those which may lead to disqualification; compare perspectives between TPDs/experts and trainees regarding the selection process and critical trainee characteristics; and investigate the general approach and satisfaction regarding current application process for ERCP/EUS training.

Methods  We conducted a web-based survey to collect general opinion and data regarding the application process and trainee selection and disqualification from training. European TPDs/experts and trainees were invited to participate.

Results  Thirty-six TPDs/experts and 25 trainees from 18 countries responded. The application process is mainly driven by individual request (86.1 %). Almost half of TPDs/experts felt only moderately (38.9 %) to slightly (8.3 %) satisfied with the current application process. TPDs/experts value a diversity of trainee characteristics, but mainly traits such as “honesty,” “being a team player,” and “self-awareness” (72.2 % each). Technical skills ranked seventh as considered “extremely important.” “Disregard for patient welfare” and “lack of work ethic” were the most common reasons for disqualification. TPDs/experts and trainees agreed in most questions.

Conclusions  This survey outlines trainee selection criteria for ERCP/EUS training. Non-technical skills are the most valued by TPDs/experts. While knowledge and technical expertise are clearly important, understanding that professional attitudes are highly regarded may help direct the application process more effectively.

Simulator-based training method in gastrointestinal endoscopy training and currently available simulators

The apprenticeship-based training method (ABTM) is highly effective for gastrointestinal (GI) endoscopic training. However, the conventional ABTM has significant issues. Although many supplementary training methods (TMs) have been developed and utilized, they cannot entirely replace the ABTM, which remains the major TM strategy. Currently, new TM construction is crucial and necessary due to financial constraints, difficulty of obtaining sufficient training time due to patient safety-related regulations, and catastrophic damage caused by disasters such as the coronavirus disease 2019 pandemic. The simulator-based TM (SBTM) is widely accepted as an alternative to the ABTM, owing to the SBTM's advantages. Since the 1960s, many GI endoscopy training simulators have been developed and numerous studies have been published on their effectiveness. While previous studies have focused on the simulator's validity, this review focused on the accessibility of simulators that were introduced by the end of 2021. Although the current SBTM is effective in GI endoscopic education, extensive improvements are needed to replace the ABTM. Incorporating simulator-incorporated TMs into an improved ABTM is an attempt to overcome the incompleteness of the current SBTM. Until a new simulator is developed to replace the ABTM, it is desirable to operate a simulator-integrated and well-coordinated TM that is suitable for each country and institution.

Development of the "Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills" (TEAM-ENTS) behavioral marker system

BACKGROUND

Non-technical skills (NTS) are integral to team performance and subsequent quality and safety of care. Behavioral marker systems (BMSs) are now increasingly used in healthcare to support the training and assessment of team NTS. Within gastrointestinal endoscopy, this is an area of novel research. The aims of this study were to define the core relevant NTS for endoscopy teams and develop a preliminary framework for a team-based BMS known as TEAM-ENTS (Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills).

METHODS

This study was conducted in two phases. In phase 1, a literature review of team-based BMSs was performed to inform an interview study of core endoscopy team members. Cognitive task analysis was used to break down the NTS relevant to endoscopy teams. Framework analysis generated the structure for the preliminary TEAM-ENTS framework. In phase 2, a modified Delphi process was undertaken to refine the items of the framework.

RESULTS

Seven consultant endoscopists and six nurses were interviewed. The final coding framework consisted of 88 codes grouped into five overarching categories. In total, 58 participants were recruited to the Delphi panel. In the first round, nine elements and 37 behavioral descriptors did not meet consensus. Following item adjustment, merging and deletion, all remaining items met consensus thresholds after the second round. The refined TEAM-ENTS BMS consists of five categories, 16 elements, and 47 behavioral descriptors.

CONCLUSIONS

The refined TEAM-ENTS behavioral marker system was developed to reflect the core NTS relevant to endoscopy teams. Future studies will aim to fully validate this tool.

Simulation-based mastery learning in gastrointestinal endoscopy training

Simulation-based mastery learning (SBML) is an emerging form of competency-based training that has been proposed as the next standard method for procedural task training, including that in gastrointestinal endoscopy. Current basic gastrointestinal endoscopy training relies on the number of procedures performed, and it has been criticized for its lack of objective standards that result in variable skills among trainees and its association with patient safety risk. Thus, incorporating simulators into a competency-based curriculum seems ideal for gastrointestinal endoscopy training. The curriculum for SBML in gastrointestinal endoscopy is currently being developed and has promising potential to translate into the clinical performance. Unlike the present apprenticeship model of “see one, do one, teach one,” SBML integrates a competency-based curriculum with specific learning objectives alongside simulation-based training. This allows trainees to practice essential skills repeatedly, receive feedback from experts, and gradually develop their abilities to achieve mastery. Moreover, trainees and trainers need to understand the learning targets of the program so that trainees can focus their learning on the necessary skills and trainers can provide structured feedback based on the expected outcomes. In addition to learning targets, an assessment plan is essential to provide trainees with future directions for their improvement and ensure patient safety by issuing a passing standard. Finally, the SBML program should be planned and managed by a specific team and conducted within a developed and tested curriculum. This review discusses the current state of gastrointestinal endoscopy training and the role of SBML in that field.

JAG consensus statements for training and certification in oesophagogastroduodenoscopy

INTRODUCTION

Training and quality assurance in oesophagogastroduodenoscopy (OGD) is important to ensure competent practice. A national evidence-based review was undertaken to update and develop standards and recommendations for OGD training and certification.

METHODS

Under the oversight of the Joint Advisory Group (JAG), a modified Delphi process was conducted with stakeholder representation from British Society of Gastroenterology, Association of Upper Gastrointestinal Surgeons, trainees and trainers. Recommendations on OGD training and certification were formulated following literature review and appraised using Grading of Recommendations Assessment, Development and Evaluation. These were subjected to electronic voting to achieve consensus. Accepted statements were incorporated into the updated certification pathway.

RESULTS

In total, 32 recommendation statements were generated for the following domains: definition of competence (4 statements), acquisition of competence (12 statements), assessment of competence (10 statements) and post-certification support (6 statements). The consensus process led to following certification criteria: (1) performing ≥250 hands-on procedures; (2) attending a JAG-accredited basic skills course; (3) attainment of relevant minimal performance standards defined by British Society of Gastroenterology/Association of Upper Gastrointestinal Surgeons of Great Britain and Ireland, (4) achieving physically unassisted D2 intubation and J-manoeuvre in ≥95% of recent procedures, (5) satisfactory performance in formative and summative direct observation of procedural skills assessments.

CONCLUSION

The JAG standards for diagnostic OGD have been updated following evidence-based consensus. These standards are intended to support training, improve competency assessment to uphold standards of practice and provide support to the newly-independent practitioner.

Endoscopy training in the UK pre-COVID-19 environment: a multidisciplinary survey of endoscopy training and the experience of reciprocal feedback

OBJECTIVE

Training in gastrointestinal endoscopy in the UK occurs predominantly in a real world one-to-one trainer to trainee interaction. Previous surveys have shown surgical and gastroenterology trainees have had mixed experiences of supervision and training, and no surveys have explored specifically the role of trainee to trainer feedback. This study aimed to explore the experience of training and of providing trainer feedback for all disciplines of endoscopy trainees.

DESIGN/METHOD

An online survey designed in collaboration with Joint Advisory Committee training committee and trainee representatives was distributed from January 2020 but was interrupted by the COVID-19 pandemic and hence terminated early.

RESULTS

There were 129 responses, including trainees from all disciplines and regions, of which 86/129 (66.7%) rated the culture in their endoscopy units favourably-either good or excellent. 65/129 (50.4%) trainees reported having one or more training lists allocated per week, with 41/129 (31.8%) reporting only ad hoc lists. 100/129 (77.5%) respondents were given feedback and 97/129 (75.2%) were provided with learning points from the list. 65/129 (50.4%) respondents reported their trainer completed a direct observation of procedure or direct observation of polypectomies. 73/129 (56.6%) respondents reported that they felt able to give feedback to their trainer, with 88/129 (68.2%) feeling they could do this accurately. Barriers to trainer feedback cited included time constraints, lack of anonymity and concerns about affecting the trainer-trainee relationship.

CONCLUSION

Overall, the training environment has improved since previous surveys. There are still issues around interdisciplinary differences with some surgical trainees finding the training environment less welcoming, and trainee perceptions of hierarchical barriers and trainer responsiveness to feedback limiting the accuracy of their feedback.

'Case of the month': a novel way to learn from endoscopy-related patient safety incidents

Patient safety incidents (PSIs) are unintended or unexpected incidents which can or do lead to patient harm. The Joint Advisory Group on Gastrointestinal Endoscopy (JAG) acknowledges that PSIs should be reviewed by endoscopy services and learning shared among staff. It is recognised that more could be done to promote shared learning as outlined by the JAG 'Improving Safety and Reducing Error in Endoscopy' strategy. The 'Case of the month' series aims to provide a broad selection of cases and subsequent learning that can be shared among services and their workforce. This review focuses on five case vignettes that highlight a variety of PSIs in endoscopy. A structured approach, based on incident analysis methodology, is applied to each case to categorise PSIs and develop learning points. Learning is directed toward the individual, team and healthcare organisation. A selection of methods to disseminate learning at local, regional and national levels are also described.

Non-technical skills curriculum incorporating simulation-based training improves performance in colonoscopy among novice endoscopists: Randomized controlled trial

BACKGROUND AND AIMS

Non-technical skills (NTS), involving cognitive, social and interpersonal skills that complement technical skills, are important for the completion of safe and efficient procedures. We investigated the impact of a simulation-based curriculum with dedicated NTS training on novice endoscopists' performance of clinical colonoscopies.

METHODS

A single-blinded randomized controlled trial was conducted at a single center. Novice endoscopists were randomized to a control curriculum or a NTS curriculum. The control curriculum involved a didactic session, virtual reality (VR) simulator colonoscopy training, and integrated scenario practice using a VR simulator, a standardized patient, and endoscopy nurse. Feedback and training were provided by experienced endoscopists. The NTS curriculum group received similar training that included a small-group session on NTS, feedback targeting NTS, and access to a self-reflective NTS checklist. The primary outcome was performance during two clinical colonoscopies, assessed using the Joint Advisory Group Direct Observation of Procedural Skills (JAG DOPS) tool.

RESULTS

Thirty-nine participants completed the study. The NTS group (n = 21) had superior clinical performance during their first (P < 0.001) and second clinical colonoscopies (P < .0.001), compared to the control group (n = 18). The NTS group performed significantly better on the VR simulator (P < 0.05) and in the integrated scenario (P < 0.05).

CONCLUSION

Our findings demonstrate that dedicated NTS training led to improved performance of clinical colonoscopies among novices.

Improving safety and reducing error in endoscopy (ISREE): a survey of UK services

BACKGROUND

The Joint Advisory Group on Gastrointestinal Endoscopy (JAG) 'Improving Safety and Reducing Error in Endoscopy' (ISREE) strategy was developed in 2018. In line with the strategy, a survey was conducted within the JAG census in 2019 to gain further insights and understanding of key safety-related areas within UK endoscopy.

METHODS

Questions were developed using the ISREE strategy as a guide and adapted by key JAG stakeholders. They were incorporated into the 2019 JAG census of UK endoscopy services. Quantitative and qualitative statistical methods were employed to analyse the results.

RESULTS

There was a 68% response rate. There was regional variability in the provision of out-of-hours GIB services (p<0.001). Across 1 month, 1535 incidents were reported across all services. There was a significantly higher proportion of reported incidents in acute services compared with others (p<0.001). Technical and training incidents were likely to be reported significantly differently to all other incident types. 74% of services have an endoscopy-specific sedation policy and 42% have a named sedation or anaesthetic lead for endoscopy. Services highlighted a desire for more anaesthetic-supported lists. Only 66% of services stated they have an effective strategy for supporting upskilling of endoscopists. Across acute services, 56% have access to human factors and endoscopic non-technical skills (ENTS) training. Patient feedback is used in several ways to improve services, develop training and promote shared learning among endoscopy users.

CONCLUSIONS

The census provides a benchmark for key safety-related characteristics of endoscopy services. These results have highlighted key areas to develop, guided by the ISREE strategy.

Development and impact of an endoscopic non-technical skills (ENTS) behavioural marker system

Background

Non-technical skills (NTS) are crucial to effective team working in endoscopy. Training in NTS has been shown to improve team performance and patient outcomes. As such, NTS training and assessment are now considered essential components of the endoscopy quality assurance process. Across the literature, other specialties have achieved this through development of behavioural marker systems (BMS). BMS provide a framework for assessing, training and measuring the NTS relevant to healthcare individuals and team. This article describes the development and impact of a novel BMS for endoscopy: the endoscopic non-technical skills (ENTS) system.

Methods

The initial NTS taxonomy for endoscopy was created through a combination of literature review, staff focus groups and semi-structured interviews, incorporating the critical decision method. Framework analysis was conducted with three individual coders and generated a skills list which formed the preliminary taxonomy. Video observation of Bowel Cancer Screening endoscopists was used to identify exemplar behaviours which were mapped to relevant skills in the NTS taxonomy. Behavioural descriptors, derived from video data, were added to form the basis of the ENTS system.

Results

A taxonomy of 33 skills in 14 separate categories were identified through framework analysis. Following video analysis and behaviour mapping, 4 overarching categories and 13 behavioural elements were identified which formed the ENTS framework. The endoscopy (directly observed procedural skills) 4-point rating scale was added to create the final ENTS system. Since its development in 2010, the ENTS system has been validated in the assessment of endoscopy for trainees nationally. ENTS informs a number of training initiatives, including a national strategy to improve NTS for all endoscopists.

Conclusions

The ENTS system is a clinically relevant tool, validated for use in trainee assessment. The use of ENTS will be important to the future of training and quality assurance in endoscopy.

Multiprofessional in situ simulation is an effective method of identifying latent patient safety threats on the gastroenterology ward

OBJECTIVE

In situ simulation (ISS) is an effective training method for multiprofessional teams dealing with emergencies in high pressured environments. A regular ISS programme was organised for the multiprofessional gastroenterology team with a primary objective of identifying, classifying and addressing latent patient safety threats and secondary objectives of improving team confidence and individual role recognition.

METHOD

22 unannounced ISS sessions (averaging approximately one session every 6 weeks and four participants per session) were conducted between February 2017 and August 2019 involving multiprofessional team members. The sessions centred around the following four common gastrointestinal emergency scenarios: massive upper gastrointestinal haemorrhage; biliary sepsis (cholangitis) and shock; postendoscopic retrograde cholangiopancreatography complications including perforation and cardiac arrest. Following the simulation, the faculty, which included nurses and doctors, facilitated a structured debrief session and action plan to identify and address latent errors.

RESULTS

96 participants from nursing, medical, physician associate and pharmacy backgrounds took part in the simulation programme. Analysis of collected latent safety threats identified the following four themes: education and training; equipment; medication and team working. Analysis of anonymously completed questionnaires identified that 95% of participants had a perceived better understanding of their role and 86% felt more confident in assessing an unwell patient. 96% of participants felt comfortable during the debrief.

CONCLUSION

ISS provides a unique opportunity to train the multiprofessional gastroenterology team in their own high-pressured environment, helping identify and address latent patient safety threats and improve perceived participant confidence and role recognition.

The emerging literature for the triad of health informatics, healthcare quality and safety, and healthcare simulation

The areas of health informatics, healthcare quality and safety, and healthcare simulation are often thought of as separate domains. The purpose of this position paper is to report on the interdependence that is emerging as an important triad across the healthcare/health system continuum. A qualitative review of 24 studies suggests the interdependence of health informatics, healthcare quality and safety, and healthcare simulation reaches much broader than traditional utilisation of simulation. We suggest ways that organisations can take advantage of the interdependence of this triad across a broader variety of healthcare environments, including teamwork, communication, and complex system relationships. In conclusion, the reviewed 24 studies suggest that the research in the triad focuses on simulation education and computerised simulation, and when coupled with health informatics, bears greater strength on quality improvement or patient safety.

Training in therapeutic endoscopy: meeting present and future challenges

Medical endoscopy trainees face numerous, often conflicting demands on their time. This can result in suboptimal endoscopy training and in difficulty achieving certification in basic endoscopy within the existing 5-year training programme. Endoscopic management of acute gastrointestinal bleeding and basic polypectomy are integral to basic service provision. Competence in these and other therapeutic procedures, including dealing with complications, is currently acquired opportunistically, or through experiential independent practice. This article proposes several potential solutions that may help with endotherapy training in the current UK training programmes. It also addresses issues relating to speciality training when reduced to 4 years in 2022. Advanced endotherapy training needs to be optimised by understanding how to select individuals with the appropriate skills and how to accelerate therapeutic training at the appropriate time. Training programmes will need to adapt and can learn from countries where the pathway is more developed and established. Future training will include a dedicated subspeciality training programme for advanced therapy with competitive entry. Advanced therapy training will be matched to service needs. Scoring systems for case complexity integrated with regional and supraregional networks, would allow referral of selected cases to the most appropriate specialised units.