51
|
Translation, cultural adaptation and content re-validation of the observational teamwork assessment for surgery tool. Int J Surg 2014; 12:1390-402. [PMID: 25462706 DOI: 10.1016/j.ijsu.2014.10.001] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2013] [Revised: 09/22/2014] [Accepted: 10/04/2014] [Indexed: 11/22/2022]
Abstract
Background. Poor teamwork and nontechnical skill performance are increasingly recognized as important contributing factors to errors and adverse events in the operating room. Assessment of these safety critical skills is important to facilitate improvement, however there are no tools available to assess these safety skills in Latin America. This study aimed to translate, culturally adapt and content validate the Observational Teamwork Assessment for Surgery (OTAS) tool for use in Latin America. Methods. A multi-phase, multi-method study was conducted: Phase 1: translation and back-translation; Phase 2: content validity assessed via expert consensus; Phase 3: inter-rater reliability assessed via real-time observation in 98 general surgical procedures using OTAS-S. Results. The first change in OTAS-S, was to distinguish between the surgical nurses and scrub technicians (both OR team members are captured in the nursing sub-team in the original OTAS). OTAS-S consists of 168 exemplar behaviors: 60/114 identical to the exemplars listed in the original OTAS tool, 48/114 original exemplars underwent minor modifications, 13 were duplicated (to account for the additional sub-team distinguished in OTAS-S), 6 original exemplars were removed, and 47 new exemplar behaviors were added. Inter-observer agreement was substantial (KW = 0.602; IC: 0.581-0.620). The calculated KW by phase, behaviors and teams were between 0.534 and 0.678. Conclusions. The study provides a content validated teamwork assessment tool for use within Colombian operating rooms and potentially Latin-American. OTAS-S can be used to assess the quality of teamwork in ORs, facilitate structured debriefing and thus improve patient safety and reduce team-related errors.
Collapse
|
52
|
Michinov E, Jamet E, Dodeler V, Haegelen C, Jannin P. Assessing neurosurgical non-technical skills: an exploratory study of a new behavioural marker system. J Eval Clin Pract 2014; 20:582-8. [PMID: 24798683 DOI: 10.1111/jep.12152] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 03/31/2014] [Indexed: 12/17/2022]
Abstract
RATIONALE, AIMS AND OBJECTIVES The management of non-technical skills is a major factor affecting teamwork quality and patient safety. This article presents a behavioural marker system for assessing neurosurgical non-technical skills (BMS-NNTS). We tested the BMS during deep brain stimulation surgery. METHOD We developed the BMS in three stages. First, we drew up a provisional assessment tool based on the literature and observation tools developed for other surgical specialties. We then analysed videos made in an operating room (OR) during deep brain stimulation operations in order to ensure there were no significant omissions from the skills list. Finally, we used five videos of operations to identify the behavioural markers of non-technical skills in verbal communications. RESULTS Analyses of more than six hours of observations revealed 3515 behaviours from which we determined the neurosurgeon's non-technical skills behaviour pattern. The neurosurgeon frequently engaged in explicit coordination, situation awareness and leadership behaviours. In addition, the neurosurgeon's behaviours differed according to the stage of the operation and the OR staff members with whom she was communicating. CONCLUSIONS Our behavioural marker system provides a structured approach to assessing non-technical skills in the field of neurosurgery. It can also be transferred to other surgical specialties and used in surgeon training curricula.
Collapse
Affiliation(s)
- Estelle Michinov
- Department of Psychology, CRPCC (E.A. 1285), University of Rennes 2, Rennes, France
| | | | | | | | | |
Collapse
|
53
|
Passauer-Baierl S, Hull L, Miskovic D, Russ S, Sevdalis N, Weigl M. Re-validating the Observational Teamwork Assessment for Surgery tool (OTAS-D): cultural adaptation, refinement, and psychometric evaluation. World J Surg 2014; 38:305-13. [PMID: 24154575 DOI: 10.1007/s00268-013-2299-8] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
BACKGROUND The nontechnical and team skills of surgical teams are critical for safety and efficiency in the operating room. Assessment of nontechnical and team skills can facilitate improvement by encouraging both self-reflection and team reflection, identifying training needs, and informing operating room (OR) team training approaches. The observational teamwork assessment for surgery (OTAS) tool is a well-validated and robust tool for capturing teamwork in the operating room. The aims of the present study were to systematically adapt and refine the OTAS for German-speaking OR staff and to test the adapted assessment tool (OTAS-D) for psychometric properties and metric equivalence. METHODS The study was carried out in three stages: at stage 1, OTAS was translated into German. At stage 2, experienced German OR experts (surgeons, OR nurses, anesthetists) were interviewed. At stage 3, two blinded assessors observed 11 general surgical operations (general surgical and vascular procedures) and interrater reliability was tested for refined OTAS-D behavioral exemplars and scorings. RESULTS The German OR experts confirmed the applicability and content validity of the vast majority of translated behavioral exemplars. After their evaluation, 32 items were changed slightly, six were changed substantially, and one item was added. During observations, perfect and substantial interobserver agreement was found for 77 behavioral exemplars (67.1 % of the items, kappa coefficient >0.60). Rating at all OTAS behaviors showed acceptable levels of reliability (intraclass correlation coefficients >0.72). CONCLUSIONS The OTAS-D is a tool for valid and reliable assessment of nontechnical skills that contribute to safe and effective surgical performance in ORs staffed by German-speaking professionals. Furthermore, our study serves as an example for systematically adapting and customizing well-established observational tools across different healthcare environments.
Collapse
Affiliation(s)
- Stefanie Passauer-Baierl
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig-Maximilians-University Munich, Ziemssentr. 1, 80336, Munich, Germany,
| | | | | | | | | | | |
Collapse
|
54
|
Khajuria A, Darzi A, Sevdalis N. Inter-professional training to enhance surgical safety: are we missing an opportunity? Br J Hosp Med (Lond) 2014; 75:414. [PMID: 25040528 DOI: 10.12968/hmed.2014.75.7.414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- Ankur Khajuria
- Year 5 Medical Student/Professor of Surgery/Reader in Patient Safety Department of Surgery and Cancer St Mary's Hospital Imperial College London London W2 1NY
| | | | | |
Collapse
|
55
|
Seelandt JC, Tschan F, Keller S, Beldi G, Jenni N, Kurmann A, Candinas D, Semmer NK. Assessing distractors and teamwork during surgery: developing an event-based method for direct observation. BMJ Qual Saf 2014; 23:918-29. [DOI: 10.1136/bmjqs-2014-002860] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|
56
|
Erratum. Br J Hosp Med (Lond) 2014. [DOI: 10.12968/hmed.2014.75.7.414a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
57
|
Chalwin RP, Flabouris A. Utility and assessment of non-technical skills for rapid response systems and medical emergency teams. Intern Med J 2014; 43:962-9. [PMID: 23611153 DOI: 10.1111/imj.12172] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2013] [Accepted: 04/14/2013] [Indexed: 11/28/2022]
Abstract
Efforts are ongoing to improve outcomes from cardiac arrest and medical emergencies. A promising quality improvement modality is use of non-technical skills (NTS) that aim to address human factors through improvements in performance of leadership, communication, situational awareness and decision-making. Originating in the airline industry, NTS training has been successfully introduced into anaesthesia, surgery, emergency medicine and other acute medical specialities. Some aspects of NTS have already achieved acceptance for cardiac arrest teams. Leadership skills are emphasised in advanced life support training and have shown favourable results when employed in simulated and clinical resuscitation scenarios. The application of NTS in medical emergency teams as part of a rapid response system attending medical emergencies is less certain; however, observations of simulations have also shown promise. This review highlights the potential benefits of NTS competency for cardiac arrest teams and, more importantly, medical emergency teams because of the diversity of clinical scenarios encountered. Discussion covers methods to assess and refine NTS and NTS training to optimise performance in the clinical environment. Increasing attention should be applied to yielding meaningful patient and organisational outcomes from use of NTS. Similarly, implementation of any training course should receive appropriate scrutiny to refine team and institutional performance.
Collapse
Affiliation(s)
- R P Chalwin
- Intensive Care Unit, Lyell McEwin Hospital, Adelaide, South Australia, Australia.
| | | |
Collapse
|
58
|
Phitayakorn R, Minehart R, Pian-Smith MCM, Hemingway MW, Milosh-Zinkus T, Oriol-Morway D, Petrusa E. Practicality of intraoperative teamwork assessments. J Surg Res 2014; 190:22-8. [PMID: 24814765 DOI: 10.1016/j.jss.2014.04.024] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2014] [Revised: 04/04/2014] [Accepted: 04/09/2014] [Indexed: 11/15/2022]
Abstract
BACKGROUND High-quality teamwork among operating room (OR) professionals is a key to efficient and safe practice. Quantification of teamwork facilitates feedback, assessment, and improvement. Several valid and reliable instruments are available for assessing separate OR disciplines and teams. We sought to determine the most feasible approach for routine documentation of teamwork in in-situ OR simulations. We compared rater agreement, hypothetical training costs, and feasibility ratings from five clinicians and two nonclinicians with instruments for assessment of separate OR groups and teams. MATERIALS AND METHODS Five teams of anesthesia or surgery residents and OR nurses (RN) or surgical technicians were videotaped in simulations of an epigastric hernia repair where the patient develops malignant hyperthermia. Two anesthesiologists, one OR clinical RN specialist, one educational psychologist, one simulation specialist, and one general surgeon discussed and then independently completed Anesthesiologists' Non-Technical Skills, Non-Technical Skills for Surgeons, Scrub Practitioners' List of Intraoperative Non-Technical Skills, and Observational Teamwork Assessment for Surgery forms to rate nontechnical performance of anesthesiologists, surgeons, nurses, technicians, and the whole team. RESULTS Intraclass correlations of agreement ranged from 0.17-0.85. Clinicians' agreements were not different from nonclinicians'. Published rater training was 4 h for Anesthesiologists' Non-Technical Skills and Scrub Practitioners' List of Intraoperative Non-Technical Skills, 2.5 h for Non-Technical Skills for Surgeons, and 15.5 h for Observational Teamwork Assessment for Surgery. Estimated costs to train one rater to use all instruments ranged from $442 for a simulation specialist to $6006 for a general surgeon. CONCLUSIONS Additional training is needed to achieve higher levels of agreement; however, costs may be prohibitive. The most cost-effective model for real-time OR teamwork assessment may be to use a simulation technician combined with one clinical rater to allow complete documentation of all participants.
Collapse
Affiliation(s)
- Roy Phitayakorn
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts; MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts.
| | - Rebecca Minehart
- MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts; Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts
| | - May C M Pian-Smith
- MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts; Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts
| | - Maureen W Hemingway
- Department of Perioperative Services, Massachusetts General Hospital, Boston, Massachusetts; Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts
| | - Tanya Milosh-Zinkus
- MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts
| | - Danika Oriol-Morway
- MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts
| | - Emil Petrusa
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts; MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts
| |
Collapse
|
59
|
Pilot Testing of a Model for Insurer-Driven, Large-Scale Multicenter Simulation Training for Operating Room Teams. Ann Surg 2014; 259:403-10. [DOI: 10.1097/sla.0000000000000342] [Citation(s) in RCA: 107] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|
60
|
Bhangu A, Bhangu S, Stevenson J, Bowley DM. Lessons for surgeons in the final moments of Air France Flight 447. World J Surg 2014; 37:1185-92. [PMID: 23463395 DOI: 10.1007/s00268-013-1971-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
BACKGROUND All surgeons make mistakes, and learning from critical incidents may help improve performance. The present study aimed to highlight lessons for surgeons from analysis of the final moments of Air France Flight 447. All of the authors work in teams and situations where safety, technical performance, and non-technical skills are critical. This review was born out of discussions regarding the events of Flight 447; specifically, whether the airline disaster was relevant to their work, and whether they could learn anything from it. METHODS The study is based on review of the crash reports of Flight 447, which lost flight speed indication after formation of ice prevented air from entering flight speed indicators during a storm. Following a subsequent stall, the aircraft fell at a rate of >10,000 feet/min until it crashed into the Atlantic Ocean, killing 228 passengers and crew. RESULTS There were errors in decision making, reasoning, communication, and teamwork. Such non-technical skills failures have been recognized previously and can be addressed by existing non-technical skills training. A reliance on autopilot meant that the pilots were unfamiliar with high-altitude flying when the autopilot is disengaged. They were unprepared for and affected by such a sudden and serious problem; an event called "surprise and startle" by the accident investigation. The absence of the senior pilot (who was on a scheduled break) in the critical final minutes slowed error recognition and recovery. CONCLUSIONS Unintended consequences of modern safety strategies may be under-recognized and can lead to adverse events. Both simulation-based and non-simulation-based training should include "surprise and startle" events beyond the scenarios trainees might expect. Likewise, in the face of increasing reliance on modern technology, surgeons should ensure that they would be able to perform procedures in the absence of such technologies. Specific training may improve surgeons' non-technical skills, and recognition of such skills could also be used to help select future surgeons.
Collapse
Affiliation(s)
- Aneel Bhangu
- General Surgery Registrar, West Midlands Deanery, Birmingham, United Kingdom.
| | | | | | | |
Collapse
|
61
|
Harrysson I, Hull L, Sevdalis N, Darzi A, Aggarwal R. Development of a knowledge, skills, and attitudes framework for training in laparoscopic cholecystectomy. Am J Surg 2014; 207:790-6. [PMID: 24524859 DOI: 10.1016/j.amjsurg.2013.08.049] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2013] [Revised: 08/14/2013] [Accepted: 08/14/2013] [Indexed: 12/25/2022]
Abstract
BACKGROUND The implementation of duty-hour restrictions and a heightened awareness of patient safety has changed resident education and training. A new focus has been placed on high-yield training programs and simulation training has naturally grown to fill this need. METHODS This article discusses the development of a training framework, knowledge, skills, and attitudes, and the design of a surgical simulation curriculum. Five residents were recruited for a pilot study of the curriculum. RESULTS A successful framework for curriculum development was implemented using laparoscopic cholecystectomy as the example. The curriculum consisted of classroom and virtual reality simulation training and was completed in 3.1 to 4.8 hours. CONCLUSIONS The current curricula that have been developed for surgical education cover the breadth of a surgical residency well. This curriculum went beyond these curricula and developed a structured framework for surgical training, a method that can be applied to any procedure.
Collapse
Affiliation(s)
- Iliana Harrysson
- Department of Surgery and Cancer, Imperial College London, QEQM Building, Praed Street, London W2 1NY, UK.
| | - Louise Hull
- Department of Surgery and Cancer, Imperial College London, QEQM Building, Praed Street, London W2 1NY, UK
| | - Nick Sevdalis
- Department of Surgery and Cancer, Imperial College London, QEQM Building, Praed Street, London W2 1NY, UK
| | - Ara Darzi
- Department of Surgery and Cancer, Imperial College London, QEQM Building, Praed Street, London W2 1NY, UK
| | - Rajesh Aggarwal
- Department of Surgery and Cancer, Imperial College London, QEQM Building, Praed Street, London W2 1NY, UK
| |
Collapse
|
62
|
Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator. J Pediatr Surg 2014; 49:29-32; discussion 32-3. [PMID: 24439576 DOI: 10.1016/j.jpedsurg.2013.09.069] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/11/2013] [Accepted: 09/30/2013] [Indexed: 11/22/2022]
Abstract
PURPOSE A validated high fidelity simulation model would provide a safe environment to teach thoracoscopic EA/TEF repair to novices. The study purpose was to evaluate validity evidence for performance measures on an EA/TEF simulator. METHODS IRB-exempt data were collected from 12 self-reported "novice" and 8 "experienced" pediatric surgeons. Participants evaluated the EA/TEF repair simulator using survey ratings that were analyzed for test content validity evidence. Additionally, deidentified operative performances were videotaped and independently rated by two surgeons using the Objective Structured Assessment for Technical Skills (OSATS) instrument. Novice and experienced OSATS were compared with p<.05 significant. RESULTS Participants had high overall simulator ratings. Internal structure was supported by high interitem consistency (α=.95 and .96) and interrater agreement (ICC) [.52, .84] for OSATS ratings. Experienced surgeons performed at a significantly higher level than novices for all five primary and two supplemental OSATS items (p<.05). CONCLUSION Favorable participant ratings indicate the simulator is relevant to clinical practice and valuable as a learning tool. Further, performance ratings can discriminate experienced and novice performances of EA/TEF repair. These findings support the use of the simulator for performance assessment, representing the first validated measures from a simulator intended for pediatric surgical training.
Collapse
|
63
|
Youngson GG. Non-technical Aspects of Safe Surgical Performance. Patient Saf Surg 2014. [DOI: 10.1007/978-1-4471-4369-7_7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022] Open
|
64
|
Vince KG. Learning curve for new technology: climbing that hill: commentary on an article by Mikko Peltola, MSc, et al.: "Learning curve for new technology? A Nationwide register-based study of 46,363 total knee arthroplasties". J Bone Joint Surg Am 2013; 95:e1901-2. [PMID: 24306713 DOI: 10.2106/jbjs.m.01145] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Affiliation(s)
- Kelly G Vince
- Department of Orthopedic Surgery, Whangarei Hospital, Northland District Hospital, Whangarei, New Zealand
| |
Collapse
|
65
|
Stocker M, Menadue L, Kakat S, De Costa K, Combes J, Banya W, Lane M, Desai A, Burmester M. Reliability of team-based self-monitoring in critical events: a pilot study. BMC Emerg Med 2013; 13:22. [PMID: 24289232 PMCID: PMC4219174 DOI: 10.1186/1471-227x-13-22] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2013] [Accepted: 11/26/2013] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Teamwork is a critical component during critical events. Assessment is mandatory for remediation and to target training programmes for observed performance gaps. METHODS The primary purpose was to test the feasibility of team-based self-monitoring of crisis resource management with a validated teamwork assessment tool. A secondary purpose was to assess item-specific reliability and content validity in order to develop a modified context-optimised assessment tool.We conducted a prospective, single-centre study to assess team-based self-monitoring of teamwork after in-situ inter-professional simulated critical events by comparison with an assessment by observers. The Mayo High Performance Teamwork Scale (MHPTS) was used as the assessment tool with evaluation of internal consistency, item-specific consensus estimates for agreement between participating teams and observers, and content validity. RESULTS 105 participants and 58 observers completed the MHPTS after a total of 16 simulated critical events over 8 months. Summative internal consistency of the MHPTS calculated as Cronbach's alpha was acceptable with 0.712 for observers and 0.710 for participants. Overall consensus estimates for dichotomous data (agreement/non-agreement) was 0.62 (Cohen's kappa; IQ-range 0.31-0.87). 6/16 items had excellent (kappa > 0.8) and 3/16 good reliability (kappa > 0.6). Short questions concerning easy to observe behaviours were more likely to be reliable. The MHPTS was modified using a threshold for good reliability of kappa > 0.6. The result is a 9 item self-assessment tool (TeamMonitor) with a calculated median kappa of 0.86 (IQ-range: 0.67-1.0) and good content validity. CONCLUSIONS Team-based self-monitoring with the MHPTS to assess team performance during simulated critical events is feasible. A context-based modification of the tool is achievable with good internal consistency and content validity. Further studies are needed to investigate if team-based self-monitoring may be used as part of a programme of assessment to target training programmes for observed performance gaps.
Collapse
Affiliation(s)
- Martin Stocker
- NICU/PICU Children’s Hospital Lucerne, CH-6000, Lucerne 16, Switzerland
| | - Lynda Menadue
- Department of Anaesthesia, Singapore General Hospital, Outram Road, Singapore 169608, Singapore
| | - Suzan Kakat
- CICU Great Ormond Street Hospital, Great Ormond Street, WC1N 3JH, London, UK
| | - Kumi De Costa
- PICU Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK
| | - Julie Combes
- PICU Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK
| | - Winston Banya
- Epidemiology and Biostatistics, Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK
| | - Mary Lane
- Department of Anaesthesia, Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK
| | - Ajay Desai
- PICU Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK
| | | |
Collapse
|
66
|
Training faculty in nontechnical skill assessment: national guidelines on program requirements. Ann Surg 2013; 258:370-5. [PMID: 23222032 DOI: 10.1097/sla.0b013e318279560b] [Citation(s) in RCA: 70] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
OBJECTIVE To develop guidelines for a faculty training program in nontechnical skill assessment in surgery. BACKGROUND Nontechnical skills in the operating room are critical for patient safety. The successful integration of these skills into workplace-based assessment is dependent upon the availability of faculty who are able to teach and assess them. At present, no guidelines exist regarding the training requirements for such faculty in surgical contexts. METHODS The development of the guidelines was carried out in several stages: stage 1-a detailed literature review on current training for nontechnical skill assessors; stage 2-semistructured interviews with a multidisciplinary panel (consisting of clinicians and psychologists/human factors specialists) of experts in surgical nontechnical skills; and stage 3-interview findings fed into an Expert Consensus Panel (ECP) Delphi approach to establish consensus regarding training requirements for faculty assessing nontechnical skills in surgery. RESULTS The ECP agreed that training in nontechnical skill assessment should be delivered by a multidisciplinary team consisting of clinicians and psychologists/human factors specialists. The ECP reached consensus regarding who should be targeted to be trained as faculty (including proficiency and revalidation requirements). Consensus was reached on 7 essential training program content elements (including training in providing feedback/debriefing) and 8 essential methods of evaluating the effectiveness of a "train-the-trainers" program. CONCLUSIONS This study provides evidence-based guidelines that can be used to guide the development and evaluation of programs to educate faculty in the training and assessment of nontechnical skills. Uptake of these guidelines could accelerate the development of surgical expertise required for safe and high-quality patient care.
Collapse
|
67
|
Wahr JA, Prager RL, Abernathy JH, Martinez EA, Salas E, Seifert PC, Groom RC, Spiess BD, Searles BE, Sundt TM, Sanchez JA, Shappell SA, Culig MH, Lazzara EH, Fitzgerald DC, Thourani VH, Eghtesady P, Ikonomidis JS, England MR, Sellke FW, Nussmeier NA. Patient safety in the cardiac operating room: human factors and teamwork: a scientific statement from the American Heart Association. Circulation 2013; 128:1139-69. [PMID: 23918255 DOI: 10.1161/cir.0b013e3182a38efa] [Citation(s) in RCA: 167] [Impact Index Per Article: 15.2] [Reference Citation Analysis] [Key Words] [MESH Headings] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
68
|
Lamb BW, Green JSA, Benn J, Brown KF, Vincent CA, Sevdalis N. Improving decision making in multidisciplinary tumor boards: prospective longitudinal evaluation of a multicomponent intervention for 1,421 patients. J Am Coll Surg 2013; 217:412-20. [PMID: 23891067 DOI: 10.1016/j.jamcollsurg.2013.04.035] [Citation(s) in RCA: 93] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2013] [Revised: 04/04/2013] [Accepted: 04/08/2013] [Indexed: 12/24/2022]
Abstract
BACKGROUND Due to its complexity, cancer care is increasingly being delivered by multidisciplinary tumor boards (MTBs). Few studies have investigated how best to organize and run MTBs to optimize clinical decision making. We developed and evaluated a multicomponent intervention designed to improve the MTB's ability to reach treatment decisions. STUDY DESIGN We conducted a prospective longitudinal study during 16 months that evaluated MTB decision making for urological cancer patients at a university hospital in London, UK. After a baseline period, MTB improvement interventions (eg, MTBs checklist, MTB team training, and written guidance) were delivered sequentially. Outcomes measures were the MTB's ability to reach a decision, the quality of information presentation, and the quality of teamwork (as assessed by trained assessors using a previously validated observational assessment tool). The efficacy of the intervention was evaluated using multivariate analyses. RESULTS There were 1,421 patients studied between December 2009 and April 2, 2011. All outcomes improved considerably between baseline and intervention implementation: the MTB's ability to reach a decision rose from 82.2% to 92.7%, quality of information presentation rose from 29.6% to 38.3%, and quality of teamwork rose from 37.8% to 43.0%. The MTB's ability to reach a treatment decision was related to the quality of available information (r = 0.298; p < 0.05) and quality of teamwork within the MTB (r = 0.348; p < 0.05). The most common barriers to reaching clinical decisions were inadequate radiologic information (n = 77), inadequate pathologic information (n = 51), and inappropriate patient referrals (n = 21). CONCLUSIONS Multidisciplinary tumor board-delivered treatment is becoming the standard for cancer care worldwide. Our intervention is efficacious and applicable to MTBs and can improve decision making and expedite cancer care.
Collapse
Affiliation(s)
- Benjamin W Lamb
- Department of Surgery and Cancer, Imperial College London, London, UK.
| | | | | | | | | | | |
Collapse
|
69
|
Pucher PH, Darzi A, Aggarwal R. Simulation for ward processes of surgical care. Am J Surg 2013; 206:96-102. [DOI: 10.1016/j.amjsurg.2012.08.013] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2012] [Revised: 06/25/2012] [Accepted: 08/19/2012] [Indexed: 11/17/2022]
|
70
|
|
71
|
The Imperial Paediatric Emergency Training Toolkit (IPETT) for use in paediatric emergency training: Development and evaluation of feasibility and validity. Resuscitation 2013; 84:831-6. [DOI: 10.1016/j.resuscitation.2012.11.013] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2012] [Revised: 11/12/2012] [Accepted: 11/13/2012] [Indexed: 12/11/2022]
|
72
|
Sevdalis N, Hull L, Birnbach D. Improving patient safety in the operating theatre and perioperative care: obstacles, interventions, and priorities for accelerating progress. Br J Anaesth 2012; 109 Suppl 1:i3-i16. [DOI: 10.1093/bja/aes391] [Citation(s) in RCA: 86] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
|
73
|
O'Leary KJ, Boudreau YN, Creden AJ, Slade ME, Williams MV. Assessment of teamwork during structured interdisciplinary rounds on medical units. J Hosp Med 2012; 7:679-83. [PMID: 22961774 DOI: 10.1002/jhm.1970] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/04/2012] [Revised: 07/09/2012] [Accepted: 07/14/2012] [Indexed: 11/09/2022]
Abstract
BACKGROUND Interdisciplinary rounds (IDR) provide a means to assemble hospital team members and improve collaboration. Little is known about teamwork during IDR. OBJECTIVE To evaluate and characterize teamwork during IDR. DESIGN Cross-sectional observational study. SETTING Six medical units which had implemented structured interdisciplinary rounds (SIDR). MEASUREMENTS We adapted the Observational Teamwork Assessment for Surgery (OTAS) tool, a behaviorally anchored rating scale shown to be reliable and valid in surgical settings. OTAS provides scores ranging from 0 to 6 (0 = problematic behavior; 6 = exemplary behavior) across 5 domains (communication, coordination, cooperation/backup behavior, leadership, and monitoring/situational awareness) and for prespecified subteams. Two researchers conducted direct observations using the adapted OTAS tool. RESULTS We conducted 7-8 independent observations for each unit (total = 44) and 20 joint observations. Inter-rater reliability was excellent at the unit level (Spearman's rho = 0.75), and good across domains (rho = 0.53-0.68) and subteams (rho = 0.53-0.76) with the exception of the physician subteam, for which it was poor (rho = 0.35). Though teamwork scores were generally high, we found differences across units, with a median (interquartile range [IQR]) 4.5 (3.9-4.9) for the lowest and 5.4 (5.3-5.5) for the highest performing unit (P < 0.01). Domain scores differed, with leadership receiving the lowest (median [IQR] = 5.0 [4.6-5.3]), and cooperation/backup behavior and monitoring/situational awareness receiving highest scores (median [IQR] = 5.4 [5.0-5.5] and 5.4 [5.0-5.7]). Differences across subteams were of borderline significance (P = 0.05). CONCLUSIONS The adapted OTAS instrument demonstrated acceptable reliability for assessing teamwork during SIDR across units, domains, and most subteams. Variation in performance suggests a need to improve consistency of teamwork and emphasizes the importance of leadership.
Collapse
Affiliation(s)
- Kevin J O'Leary
- Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, USA.
| | | | | | | | | |
Collapse
|
74
|
Lamb BW, Sevdalis N, Benn J, Vincent C, Green JSA. Multidisciplinary Cancer Team Meeting Structure and Treatment Decisions: A Prospective Correlational Study. Ann Surg Oncol 2012; 20:715-22. [DOI: 10.1245/s10434-012-2691-x] [Citation(s) in RCA: 56] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2012] [Indexed: 12/24/2022]
|
75
|
Protecting patients from an unsafe system: the etiology and recovery of intraoperative deviations in care. Ann Surg 2012; 256:203-10. [PMID: 22750753 DOI: 10.1097/sla.0b013e3182602564] [Citation(s) in RCA: 66] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE To understand the etiology and resolution of unanticipated events in the operating room (OR). BACKGROUND The majority of surgical adverse events occur intraoperatively. The OR represents a complex, high-risk system. The influence of different human, team, and organizational/environmental factors on safety and performance is unknown. METHODS We video-recorded and transcribed 10 high-acuity operations, representing 43.7 hours of patient care. Deviations, defined as delays and/or episodes of decreased patient safety, were identified by majority consensus of a multidisciplinary team. Factors that contributed to each event and/or mitigated its impact were determined and attributed to the patient, providers, or environment/organization. RESULTS Thirty-three deviations (10 delays, 17 safety compromises, 6 both) occurred--with a mean of 1 every 79.4 minutes. These deviations were multifactorial (mean 3.1 factors). Problems with communication and organizational structure appeared repeatedly at the root of both types of deviations. Delays tended to be resolved with vigilance, communication, coordination, and cooperation, while mediation of safety compromises was most frequently accomplished with vigilance, leadership, communication, and/or coordination. The organization/environment was not found to play a direct role in compensation. CONCLUSIONS Unanticipated events are common in the OR. Deviations result from poor organizational/environmental design and suboptimal team dynamics, with caregivers compensating to avoid patient harm. Although recognized in other high-risk domains, such human resilience has not yet been described in surgery and has major implications for the design of safety interventions.
Collapse
|
76
|
Wæhle HV, Haugen AS, Søfteland E, Hjälmhult E. Adjusting team involvement: a grounded theory study of challenges in utilizing a surgical safety checklist as experienced by nurses in the operating room. BMC Nurs 2012; 11:16. [PMID: 22958326 PMCID: PMC3499446 DOI: 10.1186/1472-6955-11-16] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2011] [Accepted: 08/31/2012] [Indexed: 01/11/2023] Open
Abstract
Background Even though the use of perioperative checklists have resulted in significant reduction in postoperative mortality and morbidity, as well as improvements of important information communication, the utilization of checklists seems to vary, and perceived barriers are likely to influence compliance. In this grounded theory study we aimed to explore the challenges and strategies of performing the WHO’s Safe Surgical Checklist as experienced by the nurses appointed as checklist coordinators. Methods Grounded theory was used in gathering and analyzing data from observations of the checklist used in the operating room, in conjunction with single and focus group interviews. A purposeful sample of 14 nurse-anesthetists and operating room nurses as surgical team members in a tertiary teaching hospital participated in the study. Results The nurses’ main concern regarding checklist utilization was identified as “how to obtain professional and social acceptance within the team”. The emergent grounded theory of “adjusting team involvement” consisted of three strategies; distancing, moderating and engaging team involvement. The use of these strategies explains how they resolved their challenges. Each strategy had corresponding conditions and consequences, determining checklist compliance, and how the checklist was used. Conclusion Even though nurses seem to have a loyal attitude towards the WHO’s checklist regarding their task work, they adjusted their surgical team involvement according to practical, social and professional conditions in their work environment. This might have resulted in the incomplete use of the checklist and therefore a low compliance rate. Findings also emphasized the importance of: a) management support when implementing WHO’s Safe Surgical Checklist, and b) interprofessional education approach to local adaptation of the checklists use.
Collapse
Affiliation(s)
- Hilde Valen Wæhle
- Department of Anesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway.
| | | | | | | |
Collapse
|
77
|
Sugden C, Athanasiou T, Darzi A. What Are the Effects of Sleep Deprivation and Fatigue in Surgical Practice? Semin Thorac Cardiovasc Surg 2012. [DOI: 10.1053/j.semtcvs.2012.06.005] [Citation(s) in RCA: 34] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
78
|
Development and evaluation of a decision-based simulation for assessment of team skills. Surgery 2012; 152:152-7. [DOI: 10.1016/j.surg.2012.02.018] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2011] [Accepted: 02/13/2012] [Indexed: 11/21/2022]
|
79
|
McEvoy MD, Smalley JC, Nietert PJ, Field LC, Furse CM, Blenko JW, Cobb BG, Walters JL, Pendarvis A, Dalal NS, Schaefer JJ. Validation of a detailed scoring checklist for use during advanced cardiac life support certification. Simul Healthc 2012; 7:222-35. [PMID: 22863996 PMCID: PMC3467004 DOI: 10.1097/sih.0b013e3182590b07] [Citation(s) in RCA: 34] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
INTRODUCTION Defining valid, reliable, defensible, and generalizable standards for the evaluation of learner performance is a key issue in assessing both baseline competence and mastery in medical education. However, before setting these standards of performance, the reliability of the scores yielding from a grading tool must be assessed. Accordingly, the purpose of this study was to assess the reliability of scores generated from a set of grading checklists used by nonexpert raters during simulations of American Heart Association (AHA) Megacodes. METHODS The reliability of scores generated from a detailed set of checklists, when used by 4 nonexpert raters, was tested by grading team leader performance in 8 Megacode scenarios. Videos of the scenarios were reviewed and rated by trained faculty facilitators and a group of nonexpert raters. The videos were reviewed "continuously" and "with pauses." The grading made by 2 content experts served as the reference standard, and 4 nonexpert raters were used to test the reliability of the checklists. RESULTS Our results demonstrate that nonexpert raters are able to produce reliable grades when using the checklists under consideration, demonstrating excellent intrarater reliability and agreement with a reference standard. The results also demonstrate that nonexpert raters can be trained in the proper use of the checklist in a short amount of time, with no discernible learning curve thereafter. Finally, our results show that a single trained rater can achieve reliable scores of team leader performance during AHA Megacodes when using our checklist in a continuous mode because measures of agreement in total scoring were very strong [Lin's (Biometrics 1989;45:255-268) concordance correlation coefficient, 0.96; intraclass correlation coefficient, 0.97]. CONCLUSIONS We have shown that our checklists can yield reliable scores, are appropriate for use by nonexpert raters, and are able to be used during continuous assessment of team leader performance during the review of a simulated Megacode. This checklist may be more appropriate for use by advanced cardiac life support instructors during Megacode assessments than the current tools provided by the AHA.
Collapse
Affiliation(s)
- Matthew D. McEvoy
- Department of Anesthesia and Perioperative Medicine, Assistant Dean for Patient Safety and Simulation, Medical University of South Carolina, 167 Ashley Avenue, Suite 301, Charleston, SC 29425, 843.792.2322 (phone), 843.792.2726 (fax),
| | - Jeremy C. Smalley
- Department of Anesthesia and Perioperative Medicine, Medical University of South Carolina,
| | - Paul J. Nietert
- Department of Medicine, Division of Biostatistics and Epidemiology, Medical University of South Carolina,
| | - Larry C. Field
- Department of Anesthesia and Perioperative Medicine, Medical University of South Carolina,
| | - Cory M. Furse
- Department of Anesthesia and Perioperative Medicine, Medical University of South Carolina,
| | - John W. Blenko
- Department of Anesthesiology, University of Maryland School of Medicine,
| | - Benjamin G. Cobb
- Department of Anesthesiology & Pain Medicine, University of Washington,
| | - Jenna L. Walters
- Department of Anesthesiology, Vanderbilt University Medical Center,
| | - Allen Pendarvis
- Department of Anesthesia and Perioperative Medicine, Medical University of South Carolina,
| | - Nishita S. Dalal
- Department of Anesthesia and Perioperative Medicine, Medical University of South Carolina,
| | - John J. Schaefer
- Department of Anesthesia and Perioperative Medicine, Director of Clinical Effectiveness and Patient Safety Center, Medical University of South Carolina,
| |
Collapse
|
80
|
Abstract
Despite knowledge that most surgical adverse events occur in the operating room (OR), understanding of the intraoperative phase of care is incomplete; most studies measure surgical safety in terms of preoperative risk or postoperative morbidity and mortality. Because of the OR's complexity, human factors engineering provides an ideal methodology for studies of intraoperative safety. This article reviews models of error and resilience as delineated by human factors experts, correlating them to OR performance. Existing methodologies for studying intraoperative safety are then outlined, focusing on video-based observational research. Finally, specific human and system factors examined in the OR are detailed.
Collapse
|
81
|
Hull L, Arora S, Amaya AC, Wheelock A, Gaitán-Duarte H, Vincent C, Sevdalis N. Building global capacity for patient safety: a training program for surgical safety research in developing and transitional countries. Int J Surg 2012; 10:493-9. [PMID: 22846618 DOI: 10.1016/j.ijsu.2012.07.005] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2012] [Revised: 06/25/2012] [Accepted: 07/20/2012] [Indexed: 12/31/2022]
Abstract
BACKGROUND Recent studies show a significant rate of adverse events in hospitalized patients in developing/transitional countries--with approximately 18% of them related to surgical procedures. Understanding and preventing these errors requires adequate training in patient safety research methods--however, relevant training programs are currently lacking. We developed, delivered and evaluated a training program to address this gap. METHODS A one-day training program was developed based on the recently published WHO core competencies for patient safety research. The focus was on surgical patient safety research - including human factors, operating room (OR) teamwork, the OR environment, and safety culture. Feasibility, relevance and preliminary evaluation of the program ('proof of concept' testing) was conducted in Bogotá, Colombia in July 2011. A validated evaluation framework was utilized, assessing participants' objective knowledge, attitudes, and observational skills. RESULTS 30 postgraduate students from a range of clinical/non-clinical disciplines signed up and 17 attended the program. Participants' knowledge of surgical patient safety significantly improved upon program completion (Mean pre-course=55% vs. Mean post-course=68%, P<0.01), as did their confidence and understanding of problems and methodologies to assess OR patient safety, and teamwork issues (P<0.05). Observational skills in recognizing safety-related behaviors using OTAS (i.e., quality of teamwork) improved on qualitative evaluation. CONCLUSIONS We have developed a viable, WHO-driven training program that can be delivered to clinical and non-clinical researchers to develop their competencies and thereby build capacity in developing/transitional countries to carry out surgical safety research. All program materials are available in English and Spanish for research, training and dissemination.
Collapse
Affiliation(s)
- Louise Hull
- Centre for Patient Safety and Service Quality, Department of Surgery and Cancer, Imperial College London, St Mary's Campus, Norfolk Place, London W2 1PG, UK.
| | | | | | | | | | | | | |
Collapse
|
82
|
Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg 2012; 255:804-9. [PMID: 22367446 DOI: 10.1097/sla.0b013e31824a9a02] [Citation(s) in RCA: 58] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVES To assess the feasibility of training clinical and nonclinical novice assessors to rate teamwork behavior in the operating room with short-term structured training using the observational teamwork assessment for surgery (OTAS) tool. BACKGROUND Effective teamwork is fundamental to the delivery of optimal patient care in the operating room (OR). OTAS provides a comprehensive and robust measure of teamwork in surgery. To date, assessors with a background in psychology/human factors have been shown to be able to use OTAS reliably after training. However, the feasibility of observer training over a short timescale and accessibility to the wider clinical community (ie, OTAS use by clinicians) are yet to be empirically demonstrated. METHODS Ten general surgery cases were observed and assessed using OTAS in real-time by an expert in rating OTAS behaviors (100+ cases rated) and 4 novices: 2 psychologists and 2 surgeons. Assessors were blinded to each other's scores during observations. After each observation, scores were compared and discussed between expert and novice assessors in a debriefing session. RESULTS All novices were reliable with the expert to a acceptable degree at rating all OTAS behaviors by the end of training (intraclass correlation coefficients ≥0.68). For 3 of the 5 behaviors (communication, cooperation, and leadership), calibration improved most rapidly across the first 7 observed cases. For monitoring/situational awareness, calibration improved steadily across the 10 observed cases. For coordination, no significant improvement in calibration over time was observed because of high interrater reliability from the outset (ie, a ceiling effect). There was no significant difference between surgeons and psychologists in their calibration with the expert. CONCLUSIONS It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period. OTAS is an accessible tool that can be used robustly (ie, reliably) by assessors from both clinical and nonclinical backgrounds.
Collapse
|
83
|
Flowerdew L, Gaunt A, Spedding J, Bhargava A, Brown R, Vincent C, Woloshynowych M. A multicentre observational study to evaluate a new tool to assess emergency physicians' non-technical skills. Emerg Med J 2012; 30:437-43. [DOI: 10.1136/emermed-2012-201237] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|
84
|
McKay A, Walker ST, Brett SJ, Vincent C, Sevdalis N. Team performance in resuscitation teams: comparison and critique of two recently developed scoring tools. Resuscitation 2012; 83:1478-83. [PMID: 22561464 PMCID: PMC3500685 DOI: 10.1016/j.resuscitation.2012.04.015] [Citation(s) in RCA: 44] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2012] [Revised: 04/06/2012] [Accepted: 04/22/2012] [Indexed: 12/30/2022]
Abstract
Background and aim Following high profile errors resulting in patient harm and attracting negative publicity, the healthcare sector has begun to focus on training non-technical teamworking skills as one way of reducing the rate of adverse events. Within the area of resuscitation, two tools have been developed recently aiming to assess these skills – TEAM and OSCAR. The aims of the study reported here were:To determine the inter-rater reliability of the tools in assessing performance within the context of resuscitation. To correlate scores of the same resuscitation teams episodes using both tools, thereby determining their concurrent validity within the context of resuscitation. To carry out a critique of both tools and establish how best each one may be utilised.
Methods The study consisted of two phases – reliability assessment; and content comparison, and correlation. Assessments were made by two resuscitation experts, who watched 24 pre-recorded resuscitation simulations, and independently rated team behaviours using both tools. The tools were critically appraised, and correlation between overall score surrogates was assessed. Results Both OSCAR and TEAM achieved high levels of inter-rater reliability (in the form of adequate intra-class coefficients) and minor significant differences between Wilcoxon tests. Comparison of the scores from both tools demonstrated a high degree of correlation (and hence concurrent validity). Finally, critique of each tool highlighted differences in length and complexity. Conclusion Both OSCAR and TEAM can be used to assess resuscitation teams in a simulated environment, with the tools correlating well with one another. We envisage a role for both tools – with TEAM giving a quick, global assessment of the team, but OSCAR enabling more detailed breakdown of the assessment, facilitating feedback, and identifying areas of weakness for future training.
Collapse
Affiliation(s)
- Anthony McKay
- Department of Resuscitation and Outreach, St. Mary's Hospital, Imperial College Healthcare NHS Trust, Praed Street, London W2 1NY, UK
| | - Susanna T. Walker
- Clinical Safety Research Unit, Department of Surgery and Cancer, Imperial College London, St. Mary's Hospital Campus, 10th Floor QEQM Building, St. Mary's Hospital, Praed Street, London W2 1NY, UK
- Corresponding author.
| | - Stephen J. Brett
- Center for Perioperative Medicine and Critical Care Research, Department of Anaesthesia and Intensive Care, Hammersmith Hospital, Imperial College Healthcare NHS Trust, Du Cane Road, London W12 0HS, UK
| | - Charles Vincent
- Clinical Safety Research Unit, Department of Surgery and Cancer, Imperial College London, St. Mary's Hospital Campus, 10th Floor QEQM Building, St. Mary's Hospital, Praed Street, London W2 1NY, UK
| | - Nick Sevdalis
- Clinical Safety Research Unit, Department of Surgery and Cancer, Imperial College London, St. Mary's Hospital Campus, 10th Floor QEQM Building, St. Mary's Hospital, Praed Street, London W2 1NY, UK
| |
Collapse
|
85
|
Sevdalis N, Wong HWL, Arora S, Nagpal K, Healey A, Hanna GB, Vincent CA. Quantitative analysis of intraoperative communication in open and laparoscopic surgery. Surg Endosc 2012; 26:2931-8. [DOI: 10.1007/s00464-012-2287-3] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2011] [Accepted: 03/26/2012] [Indexed: 11/28/2022]
|
86
|
Steinemann S, Berg B, DiTullio A, Skinner A, Terada K, Anzelon K, Ho HC. Assessing teamwork in the trauma bay: introduction of a modified "NOTECHS" scale for trauma. Am J Surg 2012; 203:69-75. [PMID: 22172484 DOI: 10.1016/j.amjsurg.2011.08.004] [Citation(s) in RCA: 124] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2011] [Revised: 08/30/2011] [Accepted: 08/30/2011] [Indexed: 11/27/2022]
Abstract
BACKGROUND A modified nontechnical skills (NOTECHS) scale for trauma (T-NOTECHS) was developed to teach and assess teamwork skills of multidisciplinary trauma resuscitation teams. In this study, T-NOTECHS was evaluated for reliability and correlation with clinical performance. METHODS Interrater reliability (intraclass correlation coefficient) and correlation with the speed and completeness of resuscitation tasks were assessed during simulation-based teamwork training and during actual trauma resuscitations. RESULTS For T-NOTECHS ratings done in real time, intraclass correlation coefficients were .44 for simulated and .48 for actual resuscitations. Reliability was higher (intraclass correlation coefficient = .71) for video review of resuscitations. Better T-NOTECHS scores were correlated with better performance during simulations, evidenced by a greater number of completed resuscitation tasks (r = .50, P < .01) and faster time to completion (r = -.38, P < .05) In actual resuscitations, T-NOTECHS ratings improved after teamwork training (P < .001). Higher T-NOTECHS scores were correlated with better clinical performance, evidenced by faster resuscitation (r = -.13, P < .05) and fewer unreported resuscitation tasks (r = -.16, P < .05). CONCLUSIONS Improvement in T-NOTECHS scores after teamwork training, and correlation with clinical parameters in simulated and actual trauma resuscitations, suggest its clinical relevance. Further evaluation, aiming to improve reliability, may be warranted.
Collapse
|
87
|
Arain NA, Hogg DC, Gala RB, Bhoja R, Tesfay ST, Webb EM, Scott DJ. Construct and face validity of the American College of Surgeons/Association of Program Directors in Surgery laparoscopic troubleshooting team training exercise. Am J Surg 2012; 203:54-62. [DOI: 10.1016/j.amjsurg.2011.08.010] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2011] [Revised: 08/06/2011] [Accepted: 08/06/2011] [Indexed: 11/28/2022]
|
88
|
O'Leary KJ, Sehgal NL, Terrell G, Williams MV. Interdisciplinary teamwork in hospitals: a review and practical recommendations for improvement. J Hosp Med 2012; 7:48-54. [PMID: 22042511 DOI: 10.1002/jhm.970] [Citation(s) in RCA: 116] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/05/2011] [Revised: 07/26/2011] [Accepted: 08/08/2011] [Indexed: 11/12/2022]
Abstract
Recognizing the importance of teamwork in hospitals, senior leadership from the American College of Physician Executives (ACPE), the American Hospital Association (AHA), the American Organization of Nurse Executives (AONE), and the Society of Hospital Medicine (SHM) established the High Performance Teams and the Hospital of the Future project. This collaborative learning effort aims to redesign care delivery to provide optimal value to hospitalized patients. With input from members of this initiative, we prepared this report which reviews the literature related to teamwork in hospitals. Teamwork is critically important to provide safe and effective hospital care. Hospitals with high teamwork ratings experience higher patient satisfaction, higher nurse retention, and lower hospital costs. Elements of effective teamwork have been defined and provide a framework for assessment and improvement efforts in hospitals. Measurement of teamwork is essential to understand baseline performance, and to demonstrate the utility of resources invested to enhance it and the subsequent impact on patient care. Interventions designed to improve teamwork in hospitals include localization of physicians, daily goals of care forms and checklists, teamwork training, and interdisciplinary rounds. Though additional research is needed to evaluate the impact on patient outcomes, these interventions consistently result in improved teamwork knowledge, ratings of teamwork climate, and better understanding of patients' plans of care. The optimal approach is implementation of a combination of interventions, with adaptations to fit unique clinical settings and local culture.
Collapse
Affiliation(s)
- Kevin J O'Leary
- Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
| | | | | | | | | |
Collapse
|
89
|
Abstract
The importance of non-technical skills to surgical performance is gaining wide acceptance. This article discusses the core cognitive and social skills categories thought to underpin medical knowledge and surgical expertise, and describes the rise of non-technical skill models of assessment in surgery. Behavior rating systems such as NOTSS (Non-Technical Skills for Surgeons) have been developed to support education and assessment in this regard. We now understand more about these critical skills and how they impact surgery. The challenge in the future is to incorporate them into undergraduate teaching, postgraduate training, workplace assessment, and perhaps even selection.
Collapse
Affiliation(s)
- Steven Yule
- Department of Surgery, Harvard Medical School, 25 Shattuck Street, Boston, MA 02115, USA
| | | |
Collapse
|
90
|
Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N. The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 2011; 214:214-30. [PMID: 22200377 DOI: 10.1016/j.jamcollsurg.2011.10.016] [Citation(s) in RCA: 237] [Impact Index Per Article: 18.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2011] [Revised: 10/27/2011] [Accepted: 10/31/2011] [Indexed: 01/08/2023]
Abstract
BACKGROUND Failures in nontechnical and teamwork skills frequently lie at the heart of harm and near-misses in the operating room (OR). The purpose of this systematic review was to assess the impact of nontechnical skills on technical performance in surgery. STUDY DESIGN MEDLINE, EMBASE, PsycINFO databases were searched, and 2,041 articles were identified. After limits were applied, 341 articles were retrieved for evaluation. Of these, 28 articles were accepted for this review. Data were extracted from the articles regarding sample population, study design and setting, measures of nontechnical skills and technical performance, study findings, and limitations. RESULTS Of the 28 articles that met inclusion criteria, 21 articles assessed the impact of surgeons' nontechnical skills on their technical performance. The evidence suggests that receiving feedback and effectively coping with stressful events in the OR has a beneficial impact on certain aspects of technical performance. Conversely, increased levels of fatigue are associated with detriments to surgical skill. One article assessed the impact of anesthesiologists' nontechnical skills on anesthetic technical performance, finding a strong positive correlation between the 2 skill sets. Finally, 6 articles assessed the impact of multiple nontechnical skills of the entire OR team on surgical performance. A strong relationship between teamwork failure and technical error was empirically demonstrated in these studies. CONCLUSIONS Evidence suggests that certain nontechnical aspects of performance can enhance or, if lacking, contribute to deterioration of surgeons' technical performance. The precise extent of this effect remains to be elucidated.
Collapse
Affiliation(s)
- Louise Hull
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | | | | | | | | | | |
Collapse
|
91
|
Lamb B, Payne H, Vincent C, Sevdalis N, Green JSA. The role of oncologists in multidisciplinary cancer teams in the UK: an untapped resource for team leadership? J Eval Clin Pract 2011; 17:1200-6. [PMID: 20666886 DOI: 10.1111/j.1365-2753.2010.01507.x] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
BACKGROUND AND AIMS In the UK, cancer care is managed via multidisciplinary teams (MDT). Core members of these teams are typically surgeons, oncologists, radiologists, pathologists and clinical nurse specialists (with other health care professionals potentially present). Good teamwork, including team communication and leadership, has been shown to be a prerequisite for safe care delivery in other health care contexts, but cancer MDT team working processes are yet to be fully explored. This study aimed to assess the self-perceived contribution of oncologists to MDTs, with emphasis on their potential role as team leaders. METHODS Data were collected at the British Uro-oncology Group 6th Annual Meeting (Sep 11-12 2009, York, UK). Respondents completed various items related to their perceived contribution to MDTs, aspects of current and potential MDT leadership, team decision making in these meetings, and also demographic information. RESULTS Seventy-seven oncologists attended the meeting, of whom 61 fully completed the survey (response rate 79%). Oncologists reported that their contribution to the MDT discussion carries equal weight to those of surgeons, radiologists and pathologists. Whereas 83% of respondents reported that MDT chairmanship could rotate, only 39% reported that it does in their own MDTs. More than 90% of respondents thought that oncologists (clinical or medical) could chair these meetings, but only 25% of them had ever chaired their own MDT. CONCLUSIONS Despite a high level of contribution to MDTs and the respect of their colleagues, oncologists are not taking leadership roles within MDTs at the level that they expect. This study raises the question of whether a re-evaluation the leadership of MDTs is required with clinicians from a variety of specialities being given opportunities to develop skills necessary to lead cancer MDTs and improve team performance and ultimately cancer care.
Collapse
Affiliation(s)
- Benjamin Lamb
- Centre for Patient Safety and Service Quality, Department of Surgery and Cancer, Imperial College London, London, UK.
| | | | | | | | | |
Collapse
|
92
|
O'Leary KJ, Afsar-Manesh N, Budnitz T, Dunn AS, Myers JS. Hospital quality and patient safety competencies: development, description, and recommendations for use. J Hosp Med 2011; 6:530-6. [PMID: 22042766 DOI: 10.1002/jhm.937] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2010] [Revised: 01/24/2011] [Accepted: 04/16/2011] [Indexed: 11/09/2022]
Abstract
BACKGROUND Hospitalists are poised to have a tremendous impact on improving the quality of care for hospitalized patients. However, many hospitalists are inadequately prepared to engage in efforts to improve quality, because medical schools and residency programs have not traditionally emphasized healthcare quality and patient safety in their curricula. METHODS Through a multistep process, the Society of Hospital Medicine (SHM) Quality Improvement Education (QIE) subcommittee developed the Hospital Quality and Patient Safety (HQPS) Competencies to provide a framework for developing and assessing curricula and other professional development experiences. This article describes the development, provides definitions, and makes recommendations on the use of the HQPS Competencies. RESULTS The 8 areas of competence include: Quality Measurement and Stakeholder Interests, Data Acquisition and Interpretation, Organizational Knowledge and Leadership Skills, Patient Safety Principles, Teamwork and Communication, Quality and Safety Improvement Methods, Health Information Systems, and Patient Centeredness. Reflecting differing levels of hospitalist involvement in healthcare quality, 3 levels of expertise within each area of competence have been established: basic, intermediate, and advanced. Standards for each competency area use carefully selected action verbs to reflect educational goals for hospitalists at each level. CONCLUSIONS Formal incorporation of the HQPS Competencies into professional development programs, and innovative educational initiatives and curricula, will help provide current hospitalists and the next generations of hospitalists with the needed skills to be successful.
Collapse
Affiliation(s)
- Kevin J O'Leary
- Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois 60611, USA.
| | | | | | | | | |
Collapse
|
93
|
Arora S, Russ S, Petrides KV, Sirimanna P, Aggarwal R, Darzi A, Sevdalis N. Emotional intelligence and stress in medical students performing surgical tasks. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2011; 86:1311-1317. [PMID: 21869667 DOI: 10.1097/acm.0b013e31822bd7aa] [Citation(s) in RCA: 50] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
PURPOSE Poor stress management skills can compromise performance in the operating room, particularly in inexperienced trainees. Little is known about individual differences in managing stress. This study aimed to explore the relationship between trait emotional intelligence (EI) and objective and subjective measures of stress in medical students faced with unfamiliar surgical tasks. METHOD Seventeen medical undergraduates completed an unfamiliar laparoscopic task on a simulator during January to April 2008. Subjective stress before, during (retrospectively), and after the task was measured using the self-report State-Trait Anxiety Inventory. Objective stress was measured using continuous heart rate (HR) monitoring. Participants also completed the Trait Emotional Intelligence Questionnaire short form (TEIQue-SF). The authors computed scores for global trait EI and the TEIQue-SF four factors and carried out descriptive and correlational analyses. RESULTS The highest levels of subjective stress were reported during the task and correlated positively with trait EI as well as with the trait EI factors of well-being and emotionality. Objective stress (mean HR) during the task was positively related to the sociability factor of trait EI. Higher trait EI scores were also associated with better after-task recovery from stress experienced during the task. CONCLUSIONS Students with higher trait EI are more likely to experience stress during unfamiliar surgical scenarios but are also more likely to recover better compared with their lower-trait-EI peers. Trait EI has implications for the design of effective stress management training tailored to individual needs and potential applications to surgical trainee selection and development.
Collapse
Affiliation(s)
- Sonal Arora
- Imperial College London, London, United Kingdom
| | | | | | | | | | | | | |
Collapse
|
94
|
Sharma B, Mishra A, Aggarwal R, Grantcharov TP. Non-technical skills assessment in surgery. Surg Oncol 2011; 20:169-77. [DOI: 10.1016/j.suronc.2010.10.001] [Citation(s) in RCA: 87] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
95
|
Nestel D, Cooper S, Bryant M, Higgins V, Tabak D, Murtagh G, Barraclough B. Communication challenges in surgical oncology. Surg Oncol 2011; 20:155-61. [DOI: 10.1016/j.suronc.2010.07.006] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
|
96
|
Lamb BW, Sevdalis N, Arora S, Pinto A, Vincent C, Green JSA. Teamwork and Team Decision-making at Multidisciplinary Cancer Conferences: Barriers, Facilitators, and Opportunities for Improvement. World J Surg 2011; 35:1970-6. [DOI: 10.1007/s00268-011-1152-1] [Citation(s) in RCA: 80] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
97
|
Walker S, Brett S, McKay A, Lambden S, Vincent C, Sevdalis N. Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR): development and validation. Resuscitation 2011; 82:835-44. [PMID: 21481519 PMCID: PMC3121958 DOI: 10.1016/j.resuscitation.2011.03.009] [Citation(s) in RCA: 57] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2011] [Revised: 02/26/2011] [Accepted: 03/05/2011] [Indexed: 12/30/2022]
Abstract
Aim The aim of the study reported here was to address the need to assess and train teamwork and non-technical skills in the context of Resuscitation. Specifically, we sought to develop a tool that is feasible to use and psychometrically sound to assess team behaviours during cardiac arrest resuscitation attempts. Methods To ensure validity, reliability, and feasibility, the Observational Skill based Clinical Assessment tool for Resuscitation (OSCAR) was developed in 3 phases. A review of the literature leading to initial tool development was followed by an assessment of face and content validity, and finally a thorough reliability assessment, using Cronbach's α to assess internal consistency and intraclass correlation to assess inter-rater reliability. Results OSCAR was developed methodically, and tested for face and content validity. Cronbach's α results ranged from 0.736 to 0.965 demonstrating high internal consistency, and intraclass correlation results ranged from 0.652 to 0.911, all of which are strongly significant and indicate good inter-rater reliability. Conclusion On the basis of our results, we conclude that OSCAR is psychometrically robust, scientifically sound, and clinically relevant. We have developed the Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR) for the assessment of non-technical skills in Resuscitation teams. We propose the use of this tool in simulation and real Cardiac Arrest Resuscitation attempts to assess, guide and train non-technical skills to team members, to improve patient safety and maximise the chances of successful resuscitation.
Collapse
Affiliation(s)
- S Walker
- Department of Surgery & Cancer, Imperial College London, St Mary's Hospital Campus, 10th Floor QEQM Building, St Mary's Hospital, Praed Street, London W2 1NY, UK.
| | | | | | | | | | | |
Collapse
|
98
|
Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 2011; 212:234-243.e1-5. [PMID: 21276535 DOI: 10.1016/j.jamcollsurg.2010.11.001] [Citation(s) in RCA: 146] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2010] [Revised: 11/01/2010] [Accepted: 11/02/2010] [Indexed: 12/16/2022]
Abstract
BACKGROUND Effective teamwork is crucial for safe surgery. Failures in nontechnical and teamwork skills are frequently implicated in adverse events. The Observational Teamwork Assessment for Surgery (OTAS) tool assesses teamwork of the entire team in the operating room. Empirical testing of OTAS has yet to explore the content validity of the tool. STUDY DESIGN This was a cross-sectional observational study. Data were collected in 30 procedures by 2 trained researchers. Five teamwork behaviors were scored (ie, communication, leadership, cooperation, coordination, and monitoring) and behavior exemplar completion was recorded (phase 1). Expert operating room personnel (5 surgeons, 5 anesthesiologists, and 5 scrub nurses) assessed the content validity of the OTAS exemplar behaviors. Finally, a panel of operating room patient-safety experts refined the exemplars (phase 2). RESULTS In total, the observability (presence/absence) of 130 exemplars was assessed by 2 blinded observers in 30 general surgical cases. Observer agreement was high (Cohen's κ ≥ 0.41) for 83.85% (109 of 130) of exemplar behaviors; 60.77% (79 of 130) of exemplar behaviors were observed frequently with high observer agreement. The majority of the exemplars were rated by expert operating room practitioners and an expert panel as substantial contributors to teamwork and patient safety. Based on expert consensus, 21 behavior exemplars were removed from OTAS and an additional 23 were modified. CONCLUSIONS The exemplars of OTAS demonstrated very good content validity. Taken together with recent evidence on the construct validity of the tool, these findings demonstrate that OTAS is psychometrically robust for capturing teamwork in the operating room.
Collapse
|
99
|
|
100
|
Nurok M, Sundt TM, Frankel A. Teamwork and communication in the operating room: relationship to discrete outcomes and research challenges. Anesthesiol Clin 2011; 29:1-11. [PMID: 21295749 DOI: 10.1016/j.anclin.2010.11.012] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
The literature defining and addressing teamwork and communication is abundant; however, few studies have analyzed the relationship between measures of teamwork and communication and quantifiable outcomes. The objectives of this review are: (1) to identify studies addressing teamwork and communication in the operating room in relation to discrete measures of outcome, (2) to create a classification of studies of the relationship between teamwork and communication and outcomes, (3) to assess the implications of these studies, (4) to explore the methodological challenges of teamwork and communication studies in the perioperative setting, and (5) to suggest future research directions.studies in the perioperative setting, and (5) to suggest future research directions.
Collapse
Affiliation(s)
- Michael Nurok
- Department of Anesthesiology, Hospital for Special Surgery, 535 East 70th Street, New York, NY 10021, USA.
| | | | | |
Collapse
|