1
|
Howie EE, Ambler O, Gunn EG, Dias RD, Wigmore SJ, Skipworth RJ, Yule SJ. Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room. Ann Surg 2024; 279:973-984. [PMID: 38258573 PMCID: PMC11086675 DOI: 10.1097/sla.0000000000006211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
OBJECTIVE To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance. BACKGROUND Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons. METHODS A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context. RESULTS From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making. CONCLUSIONS The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.
Collapse
Affiliation(s)
- Emma E. Howie
- Clinical Surgery, University of Edinburgh & Royal Infirmary of Edinburgh, Edinburgh, Scotland
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
| | - Olivia Ambler
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
| | - Eilidh G.M. Gunn
- Clinical Surgery, University of Edinburgh & Royal Infirmary of Edinburgh, Edinburgh, Scotland
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
| | - Roger D. Dias
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
- Human Factors and Cognitive Engineering Lab, STRATUS Centre for Medical Simulation, Brigham & Women’s Hospital, Boston, MA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA
| | - Stephen J. Wigmore
- Clinical Surgery, University of Edinburgh & Royal Infirmary of Edinburgh, Edinburgh, Scotland
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
| | - Richard J.E. Skipworth
- Clinical Surgery, University of Edinburgh & Royal Infirmary of Edinburgh, Edinburgh, Scotland
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
| | - Steven J. Yule
- Clinical Surgery, University of Edinburgh & Royal Infirmary of Edinburgh, Edinburgh, Scotland
- Edinburgh Surgical Sabermetrics Group, University of Edinburgh, Edinburgh, Scotland
| |
Collapse
|
2
|
Khoo DW, Roscoe AJ, Hwang NC. Beyond the self: a novel framework to enhance non-technical team skills for anesthesiologists. Minerva Anestesiol 2023; 89:1115-1126. [PMID: 38019175 DOI: 10.23736/s0375-9393.23.16729-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2023]
Abstract
Human factors and non-technical skills (NTS) have been identified as essential contributors to both the propagation and prevention of medical errors in the operating room. Despite extensive study and interventions to nurture and enhance NTS in anesthesiologists, gaps to effective team practice and patient safety remain. Furthermore, the link between added NTS training and clinically significant improved outcomes has not yet been demonstrated. We performed a narrative review to summarize the literature on existing systems and initiatives used to measure and nurture NTS in the clinical operating room setting. Controlled interventions performed to nurture NTS (N.=13) were identified and compared. We comment on the body of current evidence and highlight the achievements and limitations of interventions published thus far. We then propose a novel education and training framework to further develop and enhance non-technical skills in both individual anesthesiologists and operating room teams. We use the cardiac anesthesiology environment as a starting point to illustrate its use, with clinical examples. NTS is a key component of enhancing patient safety. Effective framing of its concepts is central to apply individual characteristics and skills in team environments in the OR and achieve tangible, beneficial patient outcomes.
Collapse
Affiliation(s)
- Deborah W Khoo
- Department of Anesthesiology, Singapore General Hospital, Singapore, Singapore -
| | - Andrew J Roscoe
- Department of Anesthesiology, Singapore General Hospital, Singapore, Singapore
- Department of Cardiothoracic Anesthesia, National Heart Center Singapore, Singapore, Singapore
| | - Nian C Hwang
- Department of Anesthesiology, Singapore General Hospital, Singapore, Singapore
- Department of Cardiothoracic Anesthesia, National Heart Center Singapore, Singapore, Singapore
| |
Collapse
|
3
|
Mand N, Stibane T, Sitter H, Maier RF, Leonhardt A. Successful implementation of a rater training program for medical students to evaluate simulated pediatric emergencies. GMS JOURNAL FOR MEDICAL EDUCATION 2023; 40:Doc47. [PMID: 37560048 PMCID: PMC10407587 DOI: 10.3205/zma001629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Figures] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 03/04/2023] [Accepted: 04/20/2023] [Indexed: 08/11/2023]
Abstract
Introduction Simulation-based training is increasingly used in pediatrics to teach technical skills, teamwork, and team communication, and to improve potential deficiencies in pediatric emergency care. Team performance must be observed, analyzed, and evaluated by trained raters. The structured training of medical students for the assessment of simulated pediatric emergencies has not yet been investigated. Methods We developed a rater training program for medical students to assess guideline adherence, teamwork, and team communication in simulated pediatric emergencies. Interrater reliability was measured at each training stage using Kendall tau coefficients. Results In 10 out of 15 pairs of raters interrater reliability was moderate to high (tau>0.4), whereas it was low in the remaining 5 pairs of raters. Discussion The interrater reliability showed good agreement between medical students and expert raters at the end of the rater training program. Medical students can be successfully involved in the assessment of guideline adherence as well as teamwork and team communication in simulated pediatric emergencies.
Collapse
Affiliation(s)
- Nadine Mand
- Philipps University of Marburg, University Hospital Marburg, Department of Pediatrics, Marburg, Germany
| | - Tina Stibane
- Philipps University of Marburg, Dr. Reinfried Pohl Center for Medical Education, Marburg, Germany
| | - Helmut Sitter
- Philipps University of Marburg, Dr. Reinfried Pohl Center for Medical Education, Marburg, Germany
| | - Rolf Felix Maier
- Philipps University of Marburg, University Hospital Marburg, Department of Pediatrics, Marburg, Germany
| | - Andreas Leonhardt
- Philipps University of Marburg, University Hospital Marburg, Department of Pediatrics, Marburg, Germany
| |
Collapse
|
4
|
Calhoun AW, Scerbo MW. Preparing and Presenting Validation Studies: A Guide for the Perplexed. Simul Healthc 2022; 17:357-365. [PMID: 35470343 DOI: 10.1097/sih.0000000000000667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
SUMMARY STATEMENT Simulated environments are frequently used for learner assessment, and a wide array of assessment instruments have been created to assist with this process. It is important, therefore, that clear, compelling evidence for the validity of these assessments be established. Contemporary theory recognizes instrument validity as a unified construct that links a construct to be assessed with a population, an environment of assessment, and a decision to be made using the scores. In this article, we present a primer on 2 current frameworks (Messick and Kane), define the elements of each, present a rubric that can be used by potential authors to structure their work, and offer examples of published studies showing how each framework has been successfully used to make a validity argument. We offer this with the goal of improving the quality of validity-related publications, thereby advancing the quality of assessment in healthcare simulation.
Collapse
Affiliation(s)
- Aaron W Calhoun
- From the Division of Pediatric Critical Care (A.W.C.), Department of Pediatrics, University of Louisville School of Medicine, Louisville, KY; and Department of Psychology (M.W.S.), Old Dominion University, Norfolk, VA
| | | |
Collapse
|
5
|
Bhangu A, Notario L, Pinto RL, Pannell D, Thomas-Boaz W, Freedman C, Tien H, Nathens AB, da Luz L. Closed loop communication in the trauma bay: identifying opportunities for team performance improvement through a video review analysis. CAN J EMERG MED 2022; 24:419-425. [PMID: 35412259 PMCID: PMC9002216 DOI: 10.1007/s43678-022-00295-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Accepted: 03/09/2022] [Indexed: 01/17/2023]
Abstract
Objectives Communication among trauma team members in the trauma bay is vulnerable to errors, which may impact patient outcomes. We used the previously validated trauma-non-technical skills (T-NOTECHS) tool to identify communication gaps during patient management in the trauma bay and to inform development strategies to improve team performance. Methods Two reviewers independently assessed non-technical skills of team members through video footage at Sunnybrook Health Sciences Centre. Team performance was measured using T-NOTECHS across five domains using a five-point Likert scale (lower score indicating worse performance): (1) leadership; (2) cooperation and resource management; (3) communication and interaction; (4) assessment and decision making; (5) situation awareness/coping with stress. Secondary outcomes assessed the number of callouts, closed loop communications and parallel conversations. Results The study included 55 trauma activations. Injury severity score (ISS) was used as a measure of trauma severity. A case with an ISS score ≥ 16 was considered severe. ISS was ≥ 16 in 37% of cases. Communication and interaction scored significantly lower compared to all other domains (p < 0.0001). There were significantly more callouts and completed closed loop communications in more severe cases compared to less severe cases (p = 0.017 for both). Incomplete closed loop communications and parallel conversations were identified, irrespective of case severity. Conclusion A lower communication score was identified using T-NOTECHS, attributed to incomplete closed loop communications and parallel conversations. Through video review of trauma team activations, opportunities for improvement in communication can be identified by the T-NOTECHS tool, as well as specifically identifying callouts and closed loop communication. This process may be useful for trauma programs as part of a quality improvement program on communication skills and team performance. Supplementary Information The online version contains supplementary material available at 10.1007/s43678-022-00295-z.
Collapse
Affiliation(s)
- Avneesh Bhangu
- School of Medicine, Faculty of Health Sciences, Queen's University, Unit 505 - 91 King Street East, Kingston, ON, K7L 2Z8, Canada.
| | - Lowyl Notario
- Department of Emergency Services, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.,Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada.,Tory Regional Trauma Program and the Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada
| | - Ruxandra L Pinto
- Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| | - Dylan Pannell
- Department of Surgery, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| | - Will Thomas-Boaz
- Department of Emergency Services, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.,Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada.,Tory Regional Trauma Program and the Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada
| | - Corey Freedman
- Department of Emergency Services, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.,Tory Regional Trauma Program and the Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada
| | - Homer Tien
- Tory Regional Trauma Program and the Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada.,Department of Surgery, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.,, Ornge, Mississauga, ON, Canada
| | - Avery B Nathens
- Tory Regional Trauma Program and the Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada.,Department of Surgery, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.,Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada
| | - Luis da Luz
- Tory Regional Trauma Program and the Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada.,Department of Surgery, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| |
Collapse
|
6
|
Yokoyama K, Yamamoto G, Liu C, Sugiyama O, Santos LHO, Kuroda T. Recognition of Instrument Passing and Group Attention for Understanding Intraoperative State of Surgical Team. ADVANCED BIOMEDICAL ENGINEERING 2022. [DOI: 10.14326/abe.11.37] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
|
7
|
Araujo SEA, Kim NJ, Cendoroglo NM, Klajner S. Value of Nontechnical Skills in Minimally Invasive Surgery. Clin Colon Rectal Surg 2021; 34:131-135. [PMID: 33814993 DOI: 10.1055/s-0040-1718688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Nontechnical skills are of increasing importance in surgery and surgical training. The main studies on its impact on the safety and effectiveness of surgical interventions were mainly published in the first decade of the 2000s. Due to the significant technical complexity and great diversity of instruments for nontechnical skills evaluation, the interest in training and in measuring the impact on surgical safety has relatively decreased. However, the advent of minimally invasive surgery and its peculiar technical characteristics of sophisticated technique and constant innovation through the adoption of new materials and drugs has rekindled interest in this expertise area. In the present review, we have revisited the main instruments available to measure nontechnical skill of surgical teams and analyzed the role of the main competencies on which they are based, such as situational awareness, leadership and communication skills, and the consistency of the intraoperative decision-making process. We conclude that despite the great consensus that exists among all members of the health team on the importance of nontechnical skills for the surgical team in minimally invasive surgery, the reproducible evidence on the subject is scarce and laborious to obtain. To the extent that protecting and expanding nontechnical skills is fundamental to the path toward the high reliability of health institutions, it is possible to anticipate here the role of these institutions as promoters of continuity and new research models in this area of knowledge, especially in minimally invasive surgery, an access route to which more and more patients undergoing surgical treatment in these same institutions are submitted.
Collapse
Affiliation(s)
| | - Nam Jin Kim
- Hospital Israelita Albert Einstein, São Paulo, Brasil
| | | | | |
Collapse
|
8
|
Talamini S, Halgrimson WR, Dobbs RW, Morana C, Crivellaro S. Single port robotic radical prostatectomy versus multi-port robotic radical prostatectomy: A human factor analysis during the initial learning curve. Int J Med Robot 2020; 17:e2209. [PMID: 33320437 DOI: 10.1002/rcs.2209] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2020] [Revised: 10/20/2020] [Accepted: 10/27/2020] [Indexed: 12/19/2022]
Abstract
BACKGROUND Studies have thus far neglected to evaluate the impact of the da Vinci single port (SP) robotic platform on surgeon experience and operating room efficiency. We sought to assess the effect of the SP platform on surgeon cognitive load measures during robotic assisted laparoscopic prostatectomy (RALP). METHODS We prospectively compared the first 20 SP-RALPs performed at our institution to 20 multi-port (MP)-RALPs performed by a single experienced robotic surgeon. Three multi-dimensional assessment tools were used to evaluate mental and surgical workload, teamwork and workflow disruptions. RESULTS No statistically significant differences were found between the MP-RALP and SP-RALP cohorts when evaluated by NASA Task Load Index, Surgery Task Load Index and Observational Teamwork Assessments. CONCLUSIONS The SP robotic platform did not adversely affect human factor performance of the surgeon during RALP. Multi-institutional validation will be necessary to confirm these initial findings.
Collapse
Affiliation(s)
- Susan Talamini
- Department of Urology, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Whitney R Halgrimson
- Department of Urology, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Ryan W Dobbs
- Division of Urology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Carmelo Morana
- Department of Urology, ASST Papa Giovanni XXIII Hospital, Monastier di Treviso, Italy
| | - Simone Crivellaro
- Department of Urology, University of Illinois at Chicago, Chicago, Illinois, USA
| |
Collapse
|
9
|
Santomauro CM, Hill A, McCurdie T, McGlashan HL. Improving the Quality of Evaluation Data in Simulation-Based Healthcare Improvement Projects: A Practitioner's Guide to Choosing and Using Published Measurement Tools. Simul Healthc 2020; 15:341-355. [PMID: 32520766 PMCID: PMC7531509 DOI: 10.1097/sih.0000000000000442] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
STATEMENT Simulation is increasingly being used in healthcare improvement projects. The aims of such projects can be extremely diverse. Accordingly, the outcomes or participant attributes that need to be measured can vary dramatically from project-to-project and may include a wide range of nontechnical skills, technical skills, and psychological constructs. Consequently, there is a growing need for simulation practitioners to be able to identify suitable measurement tools and incorporate them into their work. This article provides a practical introduction and guide to the key considerations for practitioners when selecting and using such tools. It also offers a substantial selection of example tools, both to illustrate the key considerations in relation to choosing a measure (including reliability and validity) and to serve as a convenient resource for those planning a study. By making well-informed choices, practitioners can improve the quality of the data they collect, and the likelihood that their projects will succeed.
Collapse
Affiliation(s)
- Chiara M Santomauro
- From the Clinical Skills Development Service (C.M.S., A.H., T.M., H.L.M.), Metro North Hospital and Health Service; and School of Psychology (C.M.S., A.H., T.M.), The University of Queensland, Brisbane, Queensland, Australia
| | | | | | | |
Collapse
|
10
|
Ramjeeawon A, Sharrock AE, Morbi A, Martin G, Riga C, Bicknell C. Using Fully-Immersive Simulation Training with Structured Debrief to Improve Nontechnical Skills in Emergency Endovascular Surgery. JOURNAL OF SURGICAL EDUCATION 2020; 77:1300-1311. [PMID: 32317159 DOI: 10.1016/j.jsurg.2020.03.023] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Revised: 03/27/2020] [Accepted: 03/29/2020] [Indexed: 05/27/2023]
Abstract
OBJECTIVE Assess whether fully-immersive simulation training with structured debriefing of a standardized emergency thoracic endovascular aortic repair (TEVAR) scenario improves team-work performance of the lead surgeon. Secondary aims: assess whether technical skills (TS) and radiation safety behaviors (RSB) improved concurrently. DESIGN Pre-post study. SETTING UK-based training days. PARTICIPANTS General and vascular surgical trainees (n = 16). INTERVENTION(S) Fully-immersive simulation training with structured debriefing of a standardized emergency TEVAR scenario. Following standardized emergency TEVAR technical training, trainees led 2 standardized fully-immersive endovascular surgery simulations, with scripted support from a passive surgical team. A non-TS (NTS) structured debrief was delivered following simulations. NTS were assessed using the validated observational teamwork assessment for surgery tool post hoc using video recordings of simulations. TS were assessed through time taken to complete each step of the procedure, as defined during technical training. RSB were assessed through checking for presence of pre-defined actions and the length of time fluoroscopy was used during each simulation. RESULTS Total observational teamwork assessment for surgery scores improved following structured debrief (p = 0.005, median 52.55/90 vs 73.0/90), alongside all constituent domains - communication (p < 0.001, median 11.7/20 vs 16.6/20), coordination (p < 0.001, median 8.6/15 vs 13.4/15), cooperation (p < 0.001, median 13.15/20 vs 16.35/20), leadership (p < 0.001, median 8.70/15 vs 11.30/15) and monitoring (p < 0.001, median 9.85/20 vs 14.85/20). TS improved; time to complete 12 of 13 procedural steps improved (p < 0.027). Fluoroscopy time (seconds) decreased (p = 0.339, 543.6 vs 495.5), frequency lead surgeons checked the team were wearing leads increased (p = 0.125, 3 vs 7) and asked the team to step back before screening increased (p = 0.003, frequency team asked to step back/total angiography runs before = 2/36 vs after = 14/44). CONCLUSIONS fully-immersive endovascular simulation with structured debrief is a robust tool to improve NTS and TS. Incorporation into surgical training may reduce operating theatres errors, increase efficiency, and improve RSB. However effective translation into the clinical workplace must be demonstrated to see these benefits.
Collapse
Affiliation(s)
- Anoopama Ramjeeawon
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom.
| | - Anna E Sharrock
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom; Imperial Vascular Unit, Imperial College Healthcare NHS Trust, London, United Kingdom; Academic Department of Military Surgery and Trauma, Royal Centre for Defence Medicine, Birmingham, United Kingdom
| | - Abigail Morbi
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom; Imperial Vascular Unit, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - Guy Martin
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom; Imperial Vascular Unit, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - Celia Riga
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom; Imperial Vascular Unit, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - Colin Bicknell
- Department of Surgery and Cancer, Imperial College London, London, United Kingdom; Imperial Vascular Unit, Imperial College Healthcare NHS Trust, London, United Kingdom
| |
Collapse
|
11
|
Allard MA, Blanié A, Brouquet A, Benhamou D. Learning non-technical skills in surgery. J Visc Surg 2020; 157:S131-S136. [DOI: 10.1016/j.jviscsurg.2020.03.001] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
12
|
McMullan RD, Urwin R, Sunderland N, Westbrook J. Observational Tools That Quantify Nontechnical Skills in the Operating Room: A Systematic Review. J Surg Res 2020; 247:306-322. [DOI: 10.1016/j.jss.2019.10.012] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2019] [Revised: 09/18/2019] [Accepted: 10/01/2019] [Indexed: 12/14/2022]
|
13
|
Hsu SYH, Hwang MJ, Mount JK. Team Communication on Tasks Performed in Blood Pressure Clinics in Community Pharmacies. J Pharm Technol 2020; 36:10-15. [PMID: 34752530 PMCID: PMC6931161 DOI: 10.1177/8755122519872697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Background: To successfully deliver pharmacy services, pharmacists and technicians need to work as a team and have effective communication. Objectives: To measure the amount of communication between community pharmacists and their technicians during monthly blood pressure (BP) clinics. Methods: Cross-sectional survey method was used to assess the amount of communication pharmacists and technicians had on each task. Study participants were pharmacist-technician pairs (teams) from 10 intervention community pharmacies in the Team Education and Adherence Monitoring (TEAM) trial. Each team provided services designed to improve BP among hypertensive African American patients. Thirteen specific tasks were identified as essential in providing monthly BP clinics, and they were being performed by either the pharmacist or technician. At the end of the trial, each pharmacist and technician were asked to report his/her perception of discussion levels that occurred for each task. The level of team communication was summarized for each task (task-specific) and for each team (team-specific). Results: For task-specific communication, 3 teams had communication regarding pharmacist tasks and 5 teams engaged in communication for technician tasks. More communication was reported for newly developed technician tasks in the BP clinic. For team-specific communication, 2 teams reported no communication on any task, and another 2 teams reported having communication on all the tasks. Overall, pharmacy teams showed different levels of communication in this study. Conclusion: The amounts of communication between pharmacists and technicians were found to vary for different tasks and teams. This suggests that the nature of tasks and the unique dynamics existing in each pharmacy team could influence pharmacist-technician communication.
Collapse
Affiliation(s)
- Shih-Ying H. Hsu
- West Coast University, Los Angeles, CA,
USA
- University of Wisconsin, Madison, WI,
USA
| | - Monica J. Hwang
- University of Wisconsin, Madison, WI,
USA
- St John’s University Queens Campus, New
York, NY, USA
| | - Jeanine K. Mount
- University of Wisconsin, Madison, WI,
USA
- Massachusetts College of Pharmacy and
Health Sciences, Boston, MA, USA
| |
Collapse
|
14
|
Abstract
OBJECTIVE Measuring teamwork is essential in critical care, but limited observational measurement systems exist for this environment. The objective of this study was to evaluate the reliability and validity of a behavioral marker system for measuring teamwork in ICUs. DESIGN Instances of teamwork were observed by two raters for three tasks: multidisciplinary rounds, nurse-to-nurse handoffs, and retrospective videos of medical students and instructors performing simulated codes. Intraclass correlation coefficients were calculated to assess interrater reliability. Generalizability theory was applied to estimate systematic sources of variance for the three observed team tasks that were associated with instances of teamwork, rater effects, competency effects, and task effects. SETTING A 15-bed surgical ICU at a large academic hospital. SUBJECTS One hundred thirty-eight instances of teamwork were observed. Specifically, we observed 88 multidisciplinary rounds, 25 nurse-to-nurse handoffs, and 25 simulated code exercises. INTERVENTIONS No intervention was conducted for this study. MEASUREMENTS AND MAIN RESULTS Rater reliability for each overall task ranged from good to excellent correlation (intraclass correlation coefficient, 0.64-0.81), although there were seven cases where reliability was fair and one case where it was poor for specific competencies. Findings from generalizability studies provided evidence that the marker system dependably distinguished among teamwork competencies, providing evidence of construct validity. CONCLUSIONS Teamwork in critical care is complex, thereby complicating the judgment of behaviors. The marker system exhibited great potential for differentiating competencies, but findings also revealed that more context specific guidance may be needed to improve rater reliability.
Collapse
|
15
|
Baxter R, Taylor N, Kellar I, Lawton R. A qualitative positive deviance study to explore exceptionally safe care on medical wards for older people. BMJ Qual Saf 2019; 28:618-626. [PMID: 30760607 PMCID: PMC6716359 DOI: 10.1136/bmjqs-2018-008023] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2018] [Revised: 12/22/2018] [Accepted: 01/20/2019] [Indexed: 11/15/2022]
Abstract
BACKGROUND The positive deviance approach seeks to identify and learn from those who demonstrate exceptional performance. This study sought to explore how multidisciplinary teams deliver exceptionally safe care on medical wards for older people. METHODS A qualitative positive deviance study was conducted on four positively deviant and four slightly-above-average matched comparator wards, which had been identified using routinely collected NHS Safety Thermometer data. In total, 70 multidisciplinary staff participated in eight focus groups to explore staff perceptions about how their teams deliver safe patient care. A thematic analysis was conducted in two stages: first to identify the tools, processes, strategies, and cultural and social contexts that facilitated safety across all wards; and second to generate hypotheses about the characteristics that facilitated 'positively deviant' patient care. RESULTS Based on identifiable qualitative differences between the positively deviant and comparison wards, 14 characteristics were hypothesised to facilitate exceptionally safe care on medical wards for older people. This paper explores five positively deviant characteristics that healthcare professionals considered to be most salient. These included the relational aspects of teamworking, specifically regarding staff knowing one another and working together in truly integrated multidisciplinary teams. The cultural and social context of positively deviant wards was perceived to influence the way in which practical tools (eg, safety briefings and bedside boards) were implemented. CONCLUSION This study exemplifies that there are no 'silver bullets' to achieving exceptionally safe patient care on medical wards for older people. Healthcare leaders should encourage truly integrated multidisciplinary ward teams where staff know each other well and work as a team. Focusing on these underpinning characteristics may facilitate exceptional performances across a broad range of safety outcomes.
Collapse
Affiliation(s)
- Ruth Baxter
- Yorkshire Quality and Safety Research Group, Bradford Institute for Health Research, Bradford, UK
| | - Natalie Taylor
- Cancer Research Division, Cancer Council, Sydney, New South Wales, Australia
| | - Ian Kellar
- School of Psychology, University of Leeds, Leeds, UK
| | - Rebecca Lawton
- Yorkshire Quality and Safety Research Group, Bradford Institute for Health Research, Bradford, UK
- School of Psychology, University of Leeds, Leeds, UK
| |
Collapse
|
16
|
Higham H, Greig PR, Rutherford J, Vincent L, Young D, Vincent C. Observer-based tools for non-technical skills assessment in simulated and real clinical environments in healthcare: a systematic review. BMJ Qual Saf 2019; 28:672-686. [DOI: 10.1136/bmjqs-2018-008565] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2018] [Revised: 04/17/2019] [Accepted: 04/23/2019] [Indexed: 12/18/2022]
Abstract
BackgroundOver the past three decades multiple tools have been developed for the assessment of non-technical skills (NTS) in healthcare. This study was designed primarily to analyse how they have been designed and tested but also to consider guidance on how to select them.ObjectivesTo analyse the context of use, method of development, evidence of validity (including reliability) and usability of tools for the observer-based assessment of NTS in healthcare.DesignSystematic review.Data sourcesSearch of electronic resources, including PubMed, Embase, CINAHL, ERIC, PsycNet, Scopus, Google Scholar and Web of Science. Additional records identified through searching grey literature (OpenGrey, ProQuest, AHRQ, King’s Fund, Health Foundation).Study selectionStudies of observer-based tools for NTS assessment in healthcare professionals (or undergraduates) were included if they: were available in English; published between January 1990 and March 2018; assessed two or more NTS; were designed for simulated or real clinical settings and had provided evidence of validity plus or minus usability. 11,101 articles were identified. After limits were applied, 576 were retrieved for evaluation and 118 articles included in this review.ResultsOne hundred and eighteen studies describing 76 tools for assessment of NTS in healthcare met the eligibility criteria. There was substantial variation in the method of design of the tools and the extent of validity, and usability testing. There was considerable overlap in the skills assessed, and the contexts of use of the tools.ConclusionThis study suggests a need for rationalisation and standardisation of the way we assess NTS in healthcare and greater consistency in how tools are developed and deployed.
Collapse
|
17
|
Ragonese M, Di GIanfrancesco L, Bassi P, Sacco E. Psychological aptitude for surgery: The importance of non-technical skills. Urologia 2019; 86:45-51. [PMID: 30983535 DOI: 10.1177/0391560319840523] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
INTRODUCTION Psychological aptitude for surgery includes all the non-technical abilities that are necessary for the surgeons. However, differently from the other careers, these skills are not considered in the selection process and their role is definitively underestimated in the field of surgery. We perform a literature review of non-technical skills for surgery to identify their role and to understand how to train and evaluate these abilities among the surgeons. RESULTS Different methods have been presented for the evaluation and training of non-technical skills for surgeons; based on the model of aviation and anesthesia a wide range of simulated scenarios have been proposed to practice these aptitudes and abilities. Different behavioral markers systems have been developed for correct identification and definition of these skills, these can be used in the real surgical room and even learned and trained in the simulated operating theatre. CONCLUSION This article shows the importance of non-practical abilities in the surgical performance and in defining the aptitude for surgery. Learning these skills and introducing them in surgical education can be useful to improve the surgical performance.
Collapse
Affiliation(s)
- Mauro Ragonese
- Urology Clinic, Catholic University of the Sacred Heart, Agostino Gemelli University Polyclinic, Rome, Italy
| | - Luca Di GIanfrancesco
- Urology Clinic, Catholic University of the Sacred Heart, Agostino Gemelli University Polyclinic, Rome, Italy
| | - PierFrancesco Bassi
- Urology Clinic, Catholic University of the Sacred Heart, Agostino Gemelli University Polyclinic, Rome, Italy
| | - Emilio Sacco
- Urology Clinic, Catholic University of the Sacred Heart, Agostino Gemelli University Polyclinic, Rome, Italy
| |
Collapse
|
18
|
Freytag J, Stroben F, Hautz WE, Schauber SK, Kämmer JE. Rating the quality of teamwork-a comparison of novice and expert ratings using the Team Emergency Assessment Measure (TEAM) in simulated emergencies. Scand J Trauma Resusc Emerg Med 2019; 27:12. [PMID: 30736821 PMCID: PMC6368771 DOI: 10.1186/s13049-019-0591-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2018] [Accepted: 11/14/2018] [Indexed: 01/01/2023] Open
Abstract
Background Training in teamwork behaviour improves technical resuscitation performance. However, its effect on patient outcome is less clear, partly because teamwork behaviour is difficult to measure. Furthermore, it is unknown who should evaluate it. In clinical practice, experts are obliged to participate in resuscitation efforts and are thus unavailable to assess teamwork quality. Consequently, we sought to determine if raters with little clinical experience and experts provide comparable evaluations of teamwork behaviour. Methods Novice and expert raters judged teamwork behaviour during 6 emergency medicine simulations using the Teamwork Emergency Assessment Measure (TEAM). Ratings of both groups were analysed descriptively and compared with U and t tests. We used a mixed effects model to identify the proportion of variance in TEAM scores attributable to rater status and other sources. Results Twelve raters evaluated 7 teams rotating through 6 cases, for a total of 84 observations. We found no significant difference between expert and novice ratings for 7 of the 11 items of the TEAM or in the sums of all item scores. Novices rated teamwork behaviour higher on 4 items and overall. Rater status accounted for 11.1% of the total variance in scores. Conclusions Experts’ and novices’ ratings were similarly distributed, implying that raters with limited experience can provide reliable data on teamwork behaviour. Novices show a consistent, but slightly more lenient rating behaviour. Clinical studies and real-life teams may thus employ novices using a structured observational tool such as TEAM to inform their performance review and improvement. Electronic supplementary material The online version of this article (10.1186/s13049-019-0591-9) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Julia Freytag
- Simulated Patients Program, Office of the Vice Dean for Teaching and Learning, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Charitéplatz 1, 10117, Berlin, Germany
| | - Fabian Stroben
- Lernzentrum, Office of the Vice Dean for Teaching and Learning, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Charitéplatz 1, 10117, Berlin, Germany.,AG Progress Test Medizin, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Charitéplatz 1, 10117, Berlin, Germany
| | - Wolf E Hautz
- Department of Emergency Medicine, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 4, 3010, Bern, Switzerland.,Centre for Health Sciences Education, University of Oslo, Gaustadalléen 30, 0373, Oslo, Norway
| | - Stefan K Schauber
- Centre for Health Sciences Education, University of Oslo, Gaustadalléen 30, 0373, Oslo, Norway
| | - Juliane E Kämmer
- AG Progress Test Medizin, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Charitéplatz 1, 10117, Berlin, Germany. .,Center for Adaptive Rationality, Max Planck Institute for Human Development, Lentzeallee 94, 14195, Berlin, Germany.
| |
Collapse
|
19
|
Sokhanvar M, Kakemam E, Goodarzi N. Implementation of the surgical safety checklist in hospitals of Iran; operating room personnel's attitude, awareness and acceptance. Int J Health Care Qual Assur 2018; 31:609-618. [PMID: 29954264 DOI: 10.1108/ijhcqa-03-2017-0051] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Purpose The WHO Surgical Safety Checklist (SSC) has improved patient safety effectively. Despite the known benefits of applying the checklist before surgery, its implementation is less than universal in practice. The purpose of this paper is to determine the operating room personnel's attitude, their awareness and knowledge of the SSC, and to evaluate staff acceptance of the SSC (including personal beliefs). Design/methodology/approach This cross-sectional study was conducted in eight tertiary general hospitals in Tehran, Iran. Some 145 operating room personnel (surgeons, anaesthetists and nurses) were selected for the study. Data collection was carried out via a validated questionnaire in three parts which included socio-demographic, attitude, awareness and acceptance. Data were then analysed using the Kruskal-Wallis and χ2 statistical test. Findings Out of the 145 participants in the study, 92 per cent were aware of the existence of the SSC and 73.9 per cent of them were aware of the objectives of SSC. Overall, the attitude to SSC was positive. The attitude of surgeons was positive towards the impact of the SSC on safety and teamwork. Surgeons were significantly more sensitive to the barriers of SSC application compared to nurses and anaesthetists ( p=0.046). Among the three groups, nurses had the highest level of support for SSC ( p=0.001). Practical implications Despite high acceptance of the checklist among staff, there is still a gap in knowledge about when exactly the checklist should be used. Therefore, involvement of all surgical team members to complete the checklist process, support of senior managers, on-going education and training and consideration of the barriers to its implementation are all key areas that need to be taken into account. Originality/value This is the first research to examine the operating room personnel's attitude, awareness and acceptance about SSC in Iranian hospitals. The outcomes of this study provide documentation and possible justification for effective establishment of SSC in Iran and other countries.
Collapse
Affiliation(s)
- Mobin Sokhanvar
- Students' Research Committee, Tabriz University of Medical Science , Tabriz, Iran
| | - Edris Kakemam
- Iranian Centre of Excellence in Health Management, School of Management and Medical Informatics, Tabriz University of Medical Science , Tabriz, Iran.,Department of Health Management and Economics, School of Public Health, Tabriz University of Medical Science , Tabriz, Iran
| | - Narges Goodarzi
- Hospital Imam Khomeini, Social Security Organization, Arak, Iran
| |
Collapse
|
20
|
Hull L, Russ S, Sevdalis N. Systematic review of methods to quantify teamwork in the operating theatre. BJS Open 2018; 2:470-472. [PMID: 30511048 PMCID: PMC6253789 DOI: 10.1002/bjs5.92] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Affiliation(s)
- L Hull
- Centre for Implementation Science, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience King's College London London SE5 8AF UK
| | - S Russ
- Centre for Implementation Science, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience King's College London London SE5 8AF UK
| | - N Sevdalis
- Centre for Implementation Science, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience King's College London London SE5 8AF UK
| |
Collapse
|
21
|
Khan A, Doke T, Boeris D. Nurturing the Next Generation of Neurosurgeons: How Important Are Nontechnical Skills? World Neurosurg 2018; 120:e227-e233. [DOI: 10.1016/j.wneu.2018.08.036] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2018] [Revised: 08/05/2018] [Accepted: 08/06/2018] [Indexed: 11/29/2022]
|
22
|
Is Teamwork Observation in the Eye of the Beholder? Crit Care Med 2018; 46:2045-2046. [PMID: 30444808 DOI: 10.1097/ccm.0000000000003461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
23
|
Pires SMP, Monteiro SOM, Pereira AMS, Stocker JNM, Chaló DDM, Melo EMDOPD. Non-technical skills assessment scale in nursing: construction, development and validation1. Rev Lat Am Enfermagem 2018; 26:e3042. [PMID: 30208160 PMCID: PMC6136549 DOI: 10.1590/1518-8345.2383.3042] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2017] [Accepted: 05/31/2018] [Indexed: 06/26/2023] Open
Abstract
OBJECTIVES The introduction of non-technical skills during nursing education is crucial to prepare nurses for the clinical context and increase patient safety. We found no instrument developed for this purpose. to construct, develop and validate a non-technical skills assessment scale in nursing. METHOD methodological research. Based on the literature review and experience of researchers on non-technical skills in healthcare and the knowledge of the principles of crisis resource management, a list of 63 items with a five-point Likert scale was constructed. The scale was applied to 177 nursing undergraduate students. Descriptive statistics, correlations, internal consistency analysis and exploratory factor analysis were performed to evaluate the psychometric properties of the scale. RESULTS scale items presented similar values for mean and median. The maximum and the minimum values presented a good distribution amongst all response options. Most items presented a significant and positive relationship. Cronbach alpha presented a good value (0.94), and most correlations were significant and positive. Exploratory factor analysis using the Kaiser-Meyer-Olkin test showed a value of 0.849, and the Bartlett's test showed adequate sphericity values (χ2=6483.998; p=0.000). One-factor model explained 26% of the total variance. CONCLUSION non-technical skills training and its measurement could be included in undergraduate or postgraduate courses in healthcare professions, or even be used to ascertain needs and improvements in healthcare contexts.
Collapse
Affiliation(s)
- Sara Martins Pereira Pires
- MSc, Researcher, Universidade de Aveiro, Departamento de Educação e Psicologia, Aveiro, Aveiro, Portugal. Researcher, Centro de Investigação em Tecnologias e Serviços de Saúde (CINTESIS), Aveiro, Portugal
| | - Sara Otília Marques Monteiro
- PhD, Assistant Professor, Universidade de Aveiro, Departamento de Educação e Psicologia, Aveiro, Aveiro, Portugal. Researcher, Centro de Investigação em Tecnologias e Serviços de Saúde (CINTESIS), Aveiro, Portugal
| | - Anabela Maria Sousa Pereira
- PhD, Associate Professor, Universidade de Aveiro, Departamento de Educação e Psicologia, Aveiro, Aveiro, Portugal
| | - Joana Novaes Machado Stocker
- MSc, Assistant Professor, Zayed University, College of Natural and Health Sciences, Dubai, Dubai, Emiratos Árabes Unidos
| | | | | |
Collapse
|
24
|
Sendlhofer G, Lumenta DB, Pregartner G, Leitgeb K, Tiefenbacher P, Gombotz V, Richter C, Kamolz LP, Brunner G. Reality check of using the surgical safety checklist: A qualitative study to observe application errors during snapshot audits. PLoS One 2018; 13:e0203544. [PMID: 30188955 PMCID: PMC6126846 DOI: 10.1371/journal.pone.0203544] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2017] [Accepted: 08/22/2018] [Indexed: 12/03/2022] Open
Abstract
Background The WHO Surgical Safety Checklist (SSC) was established to address important safety issues and to reduce the number of surgical deaths. So far, numerous reports have demonstrated sub-optimal implementation of the SSC in practice and limited improvements in patient outcomes. Therefore, the aim of this study was to audit the SSC-practice in a real-world setting in a university hospital setting. Methods From 2015 to 2016, independent observers performed snapshot audits in operating theatres and shadowed the three phases of the SSC. Using a 4-point Likert-scale to rate the compliance on each audit day, we generated a report highlighting possible improvements and provided feedback to the operating team members. Results Audits were performed on 36 operating days (2015: n = 19; 2016: n = 17), in which a total of 136 surgical interventions were observed. Overall, the percentage of “very good compliance” improved from 2015 to 2016: for the sign-in from 52.9% to 81.2% (p = 0.141), for the team-time-out from 33.3% to 58.8% (p = 0.181), and for the sign-out from 21.4% to 41.7% (p = 0.401). The qualitative review revealed inconsistencies when applying the SSC, of which the missing documentation of an actually performed item or the wrong timing for an item was most common. Conclusion Snapshot audits revealed that SSC compliance has improved over the observed period, while its application revealed inconsistencies during the three phases of the SSC. Snapshot audits proved to be a valuable tool in the qualitative analysis of SSC compliance and gave more insight than a mere completeness check of ticks in SSC documents.
Collapse
Affiliation(s)
- Gerald Sendlhofer
- Executive Department for Quality and Risk Management, University Hospital Graz, Graz, Austria
- Research Unit for Safety in Health, c/o Division of Plastic, Aesthetic and Reconstructive Surgery, Department of Surgery, Medical University of Graz, Graz, Austria
| | - David Benjamin Lumenta
- Research Unit for Safety in Health, c/o Division of Plastic, Aesthetic and Reconstructive Surgery, Department of Surgery, Medical University of Graz, Graz, Austria
- * E-mail:
| | - Gudrun Pregartner
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria
| | - Karina Leitgeb
- Executive Department for Quality and Risk Management, University Hospital Graz, Graz, Austria
| | - Peter Tiefenbacher
- Executive Department for Quality and Risk Management, University Hospital Graz, Graz, Austria
| | - Veronika Gombotz
- Executive Department for Quality and Risk Management, University Hospital Graz, Graz, Austria
| | - Christian Richter
- Executive Department for Quality and Risk Management, University Hospital Graz, Graz, Austria
| | - Lars Peter Kamolz
- Research Unit for Safety in Health, c/o Division of Plastic, Aesthetic and Reconstructive Surgery, Department of Surgery, Medical University of Graz, Graz, Austria
| | - Gernot Brunner
- Research Unit for Safety in Health, c/o Division of Plastic, Aesthetic and Reconstructive Surgery, Department of Surgery, Medical University of Graz, Graz, Austria
| |
Collapse
|
25
|
Walters SJ, Stern C, Robertson-Malt S. The measurement of collaboration within healthcare settings: a systematic review of measurement properties of instruments. ACTA ACUST UNITED AC 2018; 14:138-97. [PMID: 27532315 DOI: 10.11124/jbisrir-2016-2159] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/31/2022]
Abstract
BACKGROUND There is a growing call by consumers and governments for healthcare to adopt systems and approaches to care to improve patient safety. Collaboration within healthcare settings is an important factor for improving systems of care. By using validated measurement instruments a standardized approach to assessing collaboration is possible, otherwise it is only an assumption that collaboration is occurring in any healthcare setting. OBJECTIVES The objective of this review was to evaluate and compare measurement properties of instruments that measure collaboration within healthcare settings, specifically those which have been psychometrically tested and validated. INCLUSION CRITERIA, TYPES OF PARTICIPANTS Participants could be healthcare professionals, the patient or any non-professional who contributes to a patient's care, for example, family members, chaplains or orderlies. The term participant type means the designation of any one participant; for example 'nurse', 'social worker' or 'administrator'. More than two participant types was mandatory. TYPES OF INTERVENTION(S)/PHENOMENA OF INTEREST The focus of this review was the validity of tools used to measure collaboration within healthcare settings. TYPES OF STUDIES The types of studies considered for inclusion were validation studies, but quantitative study designs such as randomized controlled trials, controlled trials and case studies were also eligible for inclusion. Studies that focused on Interprofessional Education, were published as an abstract only, contained patient self-reporting only or were not about care delivery were excluded. OUTCOMES The outcome of interest was validation and interpretability of the instrument being assessed and included content validity, construct validity and reliability. Interpretability is characterized by statistics such as mean and standard deviation which can be translated to a qualitative meaning. SEARCH STRATEGY The search strategy aimed to find both published and unpublished studies. A three-step search strategy was utilized in this review. The databases searched included PubMed, CINAHL, Embase, Cochrane Central Register of Controlled Trials, Emerald Fulltext, MD Consult Australia, PsycARTICLES, Psychology and Behavioural Sciences Collection, PsycINFO, Informit Health Databases, Scopus, UpToDate and Web of Science. The search for unpublished studies included EThOS (Electronic Thesis Online Service), Index to Theses and ProQuest- Dissertations and Theses. METHODOLOGICAL QUALITY The assessment of methodological quality of the included studies was undertaken using the COSMIN checklist which is a validated tool that assesses the process of design and validation of healthcare measurement instruments. DATA COLLECTION An Excel spreadsheet version of COSMIN was developed for data collection which included a worksheet for extracting participant characteristics and interpretability data. DATA SYNTHESIS Statistical pooling of data was not possible for this review. Therefore, the findings are presented in a narrative form including tables and figures to aid in data presentation. To make a synthesis of the assessments of methodological quality of the different studies, each instrument was rated by accounting for the number of studies performed with an instrument, the appraisal of methodological quality and the consistency of results between studies. RESULTS Twenty-one studies of 12 instruments were included in the review. The studies were diverse in their theoretical underpinnings, target population/setting and measurement objectives. Measurement objectives included: investigating beliefs, behaviors, attitudes, perceptions and relationships associated with collaboration; measuring collaboration between different levels of care or within a multi-rater/target group; assessing collaboration across teams; or assessing internal participation of both teams and patients.Studies produced validity or interpretability data but none of the studies assessed all validity and reliability properties. However, most of the included studies produced a factor structure or referred to prior factor analysis. A narrative synthesis of the individual study factor structures was generated consisting of nine headings: organizational settings, support structures, purpose and goals; communication; reflection on process; cooperation; coordination; role interdependence and partnership; relationships; newly created professional activities; and professional flexibility. CONCLUSIONS Among the many instruments that measure collaboration within healthcare settings, the quality of each instrument varies; instruments are designed for specific populations and purposes, and are validated in various settings. Selecting an instrument requires careful consideration of the qualities of each. Therefore, referring to systematic reviews of measurement properties of instruments may be helpful to clinicians or researchers in instrument selection. IMPLICATIONS FOR PRACTICE Systematic reviews of measurement properties of instruments are valuable in aiding in instrument selection. This systematic review may be useful in instrument selection for the measurement of collaboration within healthcare settings with a complex mix of participant types. Evaluating collaboration provides important information on the strengths and limitations of different healthcare settings and the opportunities for continuous improvement via any remedial actions initiated. IMPLICATIONS FOR RESEARCH Development of a tool that can be used to measure collaboration within teams of healthcare professionals and non-professionals is important for practice. The use of different statistical modelling techniques, such as Item Response Theory modelling and the translation of models into Computer Adaptive Tests, may prove useful. Measurement equivalence is an important consideration for future instrument development and validation. Further development of the COSMIN tool should include appraisal for measurement equivalence. Researchers developing and validating measurement tools should consider multi-method research designs.
Collapse
Affiliation(s)
- Stephen John Walters
- 1The Joanna Briggs Institute, Faculty of Health Sciences, University of Adelaide, Australia 2University of Adelaide, Australia
| | | | | |
Collapse
|
26
|
Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment. Am J Surg 2018; 216:630-635. [PMID: 29366483 DOI: 10.1016/j.amjsurg.2018.01.010] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2017] [Revised: 12/19/2017] [Accepted: 01/03/2018] [Indexed: 12/21/2022]
Abstract
BACKGROUND The Trauma NOn-TECHnical Skills (T-NOTECHS) tool has been used to assess teamwork in trauma resuscitation, but its reliability and validity for self-assessment is unknown. Our purpose was to determine the reliability and validity of self-administered T-NOTECHS in pediatric trauma resuscitation. METHODS Simulated in situ resuscitations were evaluated using T-NOTECHS in real time by experts and immediately afterwards by team members. Reliability was analyzed with linear-weighted kappa and intra-class correlation. T-NOTECHS scores were compared between expert (gold-standard) and self-assessment. RESULTS Fifteen simulations were examined. T-NOTECHS scores were similar between self- and expert assessment for leadership. Self-assessment scores were higher than expert for the other domains and total composite score. Inter-rater reliability for total score was similar between the two groups, but differences were observed in the domains. CONCLUSIONS Self-assessment is not interchangeable with expert rating when using T-NOTECHS. Future studies need to determine how self-assessment can be best utilized. LEVEL OF EVIDENCE Studies of diagnostic accuracy - Level 2.
Collapse
|
27
|
Hahlweg P, Didi S, Kriston L, Härter M, Nestoriuc Y, Scholl I. Process quality of decision-making in multidisciplinary cancer team meetings: a structured observational study. BMC Cancer 2017; 17:772. [PMID: 29149872 PMCID: PMC5693525 DOI: 10.1186/s12885-017-3768-5] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2016] [Accepted: 11/09/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The quality of decision-making in multidisciplinary team meetings (MDTMs) depends on the quality of information presented and the quality of team processes. Few studies have examined these factors using a standardized approach. The aim of this study was to objectively document the processes involved in decision-making in MDTMs, document the outcomes in terms of whether a treatment recommendation was given (none vs. singular vs. multiple), and to identify factors related to type of treatment recommendation. METHODS An adaptation of the observer rating scale Multidisciplinary Tumor Board Metric for the Observation of Decision-Making (MDT-MODe) was used to assess the quality of the presented information and team processes in MDTMs. Data was analyzed using descriptive statistics and mixed logistic regression analysis. RESULTS N = 249 cases were observed in N = 29 MDTMs. While cancer-specific medical information was judged to be of high quality, psychosocial information and information regarding patient views were considered to be of low quality. In 25% of the cases no, in 64% one, and in 10% more than one treatment recommendations were given (1% missing data). Giving no treatment recommendation was associated with duration of case discussion, duration of the MDTM session, quality of case history, quality of radiological information, and specialization of the MDTM. Higher levels of medical and treatment uncertainty during discussions were found to be associated with a higher probability for more than one treatment recommendation. CONCLUSIONS The quality of different aspects of information was observed to differ greatly. In general, we did not find MDTMs to be in line with the principles of patient-centered care. Recommendation outcome varied substantially between different specializations of MDTMs. The quality of certain information was associated with the recommendation outcome. Uncertainty during discussions was related to more than one recommendation being considered. Time constraints were found to play an important role. Some of those aspects seem modifiable, which offers possibilities for the reorganization of MDTMs.
Collapse
Affiliation(s)
- Pola Hahlweg
- Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany.
| | - Sarah Didi
- Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
| | - Levente Kriston
- Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
| | - Martin Härter
- Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
| | - Yvonne Nestoriuc
- Department of Psychosomatic Medicine and Psychotherapy, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany.,Schön Klinik Hamburg Eilbek, Hamburg, Germany
| | - Isabelle Scholl
- Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
| |
Collapse
|
28
|
Pires S, Monteiro S, Pereira A, Chaló D, Melo E, Rodrigues A. Non-technical skills assessment for prelicensure nursing students: An integrative review. NURSE EDUCATION TODAY 2017; 58:19-24. [PMID: 28818708 DOI: 10.1016/j.nedt.2017.07.015] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/07/2017] [Accepted: 07/28/2017] [Indexed: 06/07/2023]
Abstract
BACKGROUND In nursing, non-technical skills are recognized as playing an important role to increase patient safety and successful clinical outcomes (Pearson and McLafferty, 2011). Non-technical skills are cognitive and social resource skills that complement technical skills and contribute to safe and efficient task performance (Flin et al., 2008). In order to effectively provide non-technical skills training, it is essential to have an instrument to measure these skills. METHODOLOGY An online search was conducted. Articles were selected if they referred to and/or described instruments assessing non-technical skills for nurses and/or prelicensure nursing students in educational, clinical and/or simulated settings with validation evidence (inclusion criteria). RESULTS Of the 53 articles located, 26 met the inclusion criteria. Those referred to and/or described 16 instruments with validation evidence developed to assess non-technical skills in multidisciplinary teams including nurses. CONCLUSION Although articles have shown 16 valid and reliable instruments, to our knowledge, no instrument has been published or developed and validated for the assessment of non-technical skills of only nurses in general, relevant for use in high-fidelity simulation-based training for prelicensure nursing students. Therefore, there is a need for the development of such an instrument.
Collapse
Affiliation(s)
- Sara Pires
- Departamento de Educação, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal; Cintesis - Center for Health Technology and Services Research, Piso 2, edifício nascente, Centro de Investigação Médica, Faculdade de Medicina da Universidade do Porto, Rua Dr. Plácido da Costa, s/n, 4200-450 Porto, Portugal.
| | - Sara Monteiro
- Departamento de Educação, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal; Cintesis - Center for Health Technology and Services Research, Piso 2, edifício nascente, Centro de Investigação Médica, Faculdade de Medicina da Universidade do Porto, Rua Dr. Plácido da Costa, s/n, 4200-450 Porto, Portugal.
| | - Anabela Pereira
- Departamento de Educação, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal; Centro de Investigação Didáctica e Tecnologia na Formação de Formadores, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal.
| | - Daniela Chaló
- Escola Superior de Saúde da Universidade de Aveiro, Edifício 30 - Agras do Crasto, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal; SIMULA - Centro de Simulação Clínica da Universidade de Aveiro, Edifício 30 - Agras do Crasto, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal..
| | - Elsa Melo
- SIMULA - Centro de Simulação Clínica da Universidade de Aveiro, Edifício 30 - Agras do Crasto, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal.; Escola Superior de Saúde da Universidade de Aveiro, Edifício 30 - Agras do Crasto, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal.
| | - Alexandre Rodrigues
- SIMULA - Centro de Simulação Clínica da Universidade de Aveiro, Edifício 30 - Agras do Crasto, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal.; Escola Superior de Saúde da Universidade de Aveiro, Edifício 30 - Agras do Crasto, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal.
| |
Collapse
|
29
|
Nontechnical skill training and the use of scenarios in modern surgical education. Curr Opin Urol 2017; 27:330-336. [DOI: 10.1097/mou.0000000000000402] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
30
|
Randell R, Honey S, Hindmarsh J, Alvarado N, Greenhalgh J, Pearman A, Long A, Cope A, Gill A, Gardner P, Kotze A, Wilkinson D, Jayne D, Croft J, Dowding D. A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making. HEALTH SERVICES AND DELIVERY RESEARCH 2017. [DOI: 10.3310/hsdr05200] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023] Open
Abstract
BackgroundThe implementation of robot-assisted surgery (RAS) can be challenging, with reports of surgical robots being underused. This raises questions about differences compared with open and laparoscopic surgery and how best to integrate RAS into practice.ObjectivesTo (1) contribute to reporting of the ROLARR (RObotic versus LAparoscopic Resection for Rectal cancer) trial, by investigating how variations in the implementation of RAS and the context impact outcomes; (2) produce guidance on factors likely to facilitate successful implementation; (3) produce guidance on how to ensure effective teamwork; and (4) provide data to inform the development of tools for RAS.DesignRealist process evaluation alongside ROLARR. Phase 1 – a literature review identified theories concerning how RAS becomes embedded into practice and impacts on teamwork and decision-making. These were refined through interviews across nine NHS trusts with theatre teams. Phase 2 – a multisite case study was conducted across four trusts to test the theories. Data were collected using observation, video recording, interviews and questionnaires. Phase 3 – interviews were conducted in other surgical disciplines to assess the generalisability of the findings.FindingsThe introduction of RAS is surgeon led but dependent on support at multiple levels. There is significant variation in the training provided to theatre teams. Contextual factors supporting the integration of RAS include the provision of whole-team training, the presence of handpicked dedicated teams and the availability of suitably sized operating theatres. RAS introduces challenges for teamwork that can impact operation duration, but, over time, teams develop strategies to overcome these challenges. Working with an experienced assistant supports teamwork, but experience of the procedure is insufficient for competence in RAS and experienced scrub practitioners are important in supporting inexperienced assistants. RAS can result in reduced distraction and increased concentration for the surgeon when he or she is supported by an experienced assistant or scrub practitioner.ConclusionsOur research suggests a need to pay greater attention to the training and skill mix of the team. To support effective teamwork, our research suggests that it is beneficial for surgeons to (1) encourage the team to communicate actions and concerns; (2) alert the attention of the assistant before issuing a request; and (3) acknowledge the scrub practitioner’s role in supporting inexperienced assistants. It is beneficial for the team to provide oral responses to the surgeon’s requests.LimitationsThis study started after the trial, limiting impact on analysis of the trial. The small number of operations observed may mean that less frequent impacts of RAS were missed.Future workFuture research should include (1) exploring the transferability of guidance for effective teamwork to other surgical domains in which technology leads to the physical or perceptual separation of surgeon and team; (2) exploring the benefits and challenges of including realist methods in feasibility and pilot studies; (3) assessing the feasibility of using routine data to understand the impact of RAS on rare end points associated with patient safety; (4) developing and evaluating methods for whole-team training; and (5) evaluating the impact of different physical configurations of the robotic console and team members on teamwork.FundingThe National Institute for Health Research Health Services and Delivery Research programme.
Collapse
Affiliation(s)
- Rebecca Randell
- School of Healthcare, Faculty of Medicine and Health, University of Leeds, Leeds, UK
| | - Stephanie Honey
- School of Healthcare, Faculty of Medicine and Health, University of Leeds, Leeds, UK
| | - Jon Hindmarsh
- School of Management & Business, Faculty of Social Science & Public Policy, King’s College London, London, UK
| | - Natasha Alvarado
- Institute of Health and Society, Newcastle University, Newcastle upon Tyne, UK
| | - Joanne Greenhalgh
- School of Sociology and Social Policy, Faculty of Education, Social Sciences and Law, University of Leeds, Leeds, UK
| | - Alan Pearman
- Centre for Decision Research, University of Leeds, Leeds, UK
| | - Andrew Long
- School of Healthcare, Faculty of Medicine and Health, University of Leeds, Leeds, UK
| | - Alexandra Cope
- Leeds Institute of Medical Education, University of Leeds, Leeds, UK
| | - Arron Gill
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Peter Gardner
- School of Psychology, Faculty of Medicine and Health, University of Leeds, Leeds, UK
| | - Alwyn Kotze
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | - David Jayne
- Leeds Institute of Biomedical & Clinical Sciences, Faculty of Medicine and Health, University of Leeds, Leeds, UK
| | - Julie Croft
- Leeds Institute of Clinical Trials Research, Faculty of Medicine and Health, University of Leeds, Leeds, UK
| | - Dawn Dowding
- School of Nursing, Columbia University Medical Center, Columbia University, New York, NY, USA
| |
Collapse
|
31
|
Rehim SA, DeMoor S, Olmsted R, Dent DL, Parker-Raley J. Tools for Assessment of Communication Skills of Hospital Action Teams: A Systematic Review. JOURNAL OF SURGICAL EDUCATION 2017; 74:341-351. [PMID: 27771338 DOI: 10.1016/j.jsurg.2016.09.008] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2016] [Revised: 08/07/2016] [Accepted: 09/27/2016] [Indexed: 06/06/2023]
Abstract
BACKGROUND Hospital action teams comprise interdisciplinary health care providers working simultaneously to treat critically ill patients. Assessments designed to evaluate communication effectiveness or "nontechnical" performance of these teams are essential to minimize medical errors and improve team productivity. Although multiple communication tools are available, the characteristics and psychometric validity of these instruments have yet to be systematically compared. OBJECTIVE To identify assessments used to evaluate the communication or "nontechnical" performance of hospital action teams and summarize evidence to develop and validate these instruments. METHOD A literature search was conducted using MEDLINE/PubMed database to identify original articles related to assessment of communication skills in teams working in acute care medicine not exclusive to emergency room, operating room, prehospital air and ground transport, or code blue/rapid response resuscitations. RESULTS Ten communication assessment tools were identified. Six tools (60%) were designed to measure communication performance of the whole team, whereas 4 tools (40%) were created to assess individual team member's communication skills. Regardless of the type of analysis, the most commonly assessed behavior domains were Leadership, Teamwork, Communication, and Situation awareness. Only 1 of 16 articles describing a particular communication assessment tool reported all the validation criteria, other authors underreported efforts to validate their instruments. CONCLUSION A number of tools designed to measure the communication or "nontechnical" performance of hospital action teams are available. Unfortunately, limited reported validity evidence may hamper the utility of these tools in actual clinical practice until further validation studies are performed.
Collapse
Affiliation(s)
- Shady A Rehim
- Department of Surgery, University of Texas Health Science Center San Antonio, San Antonio, Texas
| | - Stephanie DeMoor
- Department of Surgery, University of Texas Health Science Center San Antonio, San Antonio, Texas
| | - Richard Olmsted
- University of Texas Health Science Center San Antonio, San Antonio, Texas
| | - Daniel L Dent
- University of Texas Health Science Center San Antonio, San Antonio, Texas
| | - Jessica Parker-Raley
- Division of Trauma, University of Texas Health Science Center San Antonio, San Antonio, Texas.
| |
Collapse
|
32
|
Strategies for increasing the feasibility of performance assessments during competency-based education: Subjective and objective evaluations correlate in the operating room. Am J Surg 2016; 214:365-372. [PMID: 27634423 DOI: 10.1016/j.amjsurg.2016.07.017] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2016] [Revised: 06/23/2016] [Accepted: 07/06/2016] [Indexed: 11/22/2022]
Abstract
BACKGROUND Competency-based education necessitates assessments that determine whether trainees have acquired specific competencies. The evidence on the ability of internal raters (staff surgeons) to provide accurate assessments is mixed; however, this has not yet been directly explored in the operating room. This study's objective is to compare the ratings given by internal raters vs an expert external rater (independent to the training process) in the operating room. METHODS Raters assessed general surgery residents during a laparoscopic cholecystectomy for their technical and nontechnical performance. RESULTS Fifteen cases were observed. There was a moderately positive correlation (rs = .618, P = .014) for technical performance and a strong positive correlation (rs = .731, P = .002) for nontechnical performance. The internal raters were less stringent for technical (mean rank 3.33 vs 8.64, P = .007) and nontechnical (mean rank 3.83 vs 8.50, P = .01) performances. CONCLUSIONS This study provides evidence to help operationalize competency-based assessments.
Collapse
|
33
|
Content Validation and Evaluation of an Endovascular Teamwork Assessment Tool. Eur J Vasc Endovasc Surg 2016; 52:11-20. [DOI: 10.1016/j.ejvs.2015.12.044] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2015] [Accepted: 12/24/2015] [Indexed: 11/21/2022]
|
34
|
Improving Escalation of Care: Development and Validation of the Quality of Information Transfer Tool. Ann Surg 2016; 263:477-86. [PMID: 25775058 DOI: 10.1097/sla.0000000000001164] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE To develop and provide validity and feasibility evidence for the QUality of Information Transfer (QUIT) tool. BACKGROUND Prompt escalation of care in the setting of patient deterioration can prevent further harm. Escalation and information transfer skills are not currently measured in surgery. METHODS This study comprised 3 phases: the development (phase 1), validation (phase 2), and feasibility analysis (phase 3) of the QUIT tool. Phase 1 involved identification of core skills needed for successful escalation of care through literature review and 33 semistructured interviews with stakeholders. Phase 2 involved the generation of validity evidence for the tool using a simulated setting. Thirty surgeons assessed a deteriorating postoperative patient in a simulated ward and escalated their care to a senior colleague. The face and content validity were assessed using a survey. Construct and concurrent validity of the tool were determined by comparing performance scores using the QUIT tool with those measured using the Situation-Background-Assessment-Recommendation (SBAR) tool. Phase 3 was conducted using direct observation of escalation scenarios on surgical wards in 2 hospitals. RESULTS A 7-category assessment tool was developed from phase 1 consisting of 24 items. Twenty-one of 24 items had excellent content validity (content validity index >0.8). All 7 categories and 18 of 24 (P < 0.05) items demonstrated construct validity. The correlation between the QUIT and SBAR tools used was strong indicating concurrent validity (r = 0.694, P < 0.001). Real-time scoring of escalation referrals was feasible and indicated that doctors currently have better information transfer skills than nurses when faced with a deteriorating patient. CONCLUSIONS A validated tool to assess information transfer for deteriorating surgical patients was developed and tested using simulation and real-time clinical scenarios. It may improve the quality and safety of patient care on the surgical ward.
Collapse
|
35
|
Havyer RD, Nelson DR, Wingo MT, Comfere NI, Halvorsen AJ, McDonald FS, Reed DA. Addressing the Interprofessional Collaboration Competencies of the Association of American Medical Colleges: A Systematic Review of Assessment Instruments in Undergraduate Medical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:865-88. [PMID: 26703415 DOI: 10.1097/acm.0000000000001053] [Citation(s) in RCA: 59] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
PURPOSE To summarize characteristics and validity evidence of tools that assess teamwork in undergraduate medical education (UME), and provide recommendations for addressing the interprofessional collaboration competencies of the Association of American Medical Colleges (AAMC). METHOD The authors conducted a systematic review, searching MEDLINE, MEDLINE In-process, CINAHL, and PsycINFO from January 1, 1979, through April 1, 2014; they searched reference lists and national meeting abstracts. They included original research reports that described a quantitative tool used to assess teamwork in UME. They abstracted characteristics and validity evidence for the tools, plus study quality, according to established frameworks. Two authors independently abstracted 25% of articles and calculated agreement. Authors then applied predefined criteria to identify tools best suited to address the AAMC's teamwork competencies. RESULTS Of 13,549 citations, 70 articles describing 64 teamwork assessment tools were included. Of these 64 tools, 27 (42%) assessed teamwork in classroom, 31 (48%) in simulation, and only 7 (11%) in actual clinical settings. The majority (47; 73%) of tools assessed medical students' teamwork in interprofessional teams. On the basis of content concordance, strength of validity evidence, generalizability of scores, and level of outcomes, four published tools were recommended to assess the AAMC's teamwork competencies: the Collaborative Healthcare Interdisciplinary Relationship Planning Scale, Readiness for Interprofessional Learning Scale, Communication and Teamwork Skills assessment, and Teamwork Mini-Clinical Evaluation Exercise. CONCLUSIONS Substantial validity evidence supports the use of several UME teamwork assessments. Four tools have been appropriately designed and sufficiently studied to constitute appropriate assessments of the AAMC's teamwork competencies.
Collapse
Affiliation(s)
- Rachel D Havyer
- R.D. Havyer is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. D.R. Nelson is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. M.T. Wingo is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. N.I. Comfere is associate professor of dermatology and of laboratory medicine and pathology, Mayo Clinic College of Medicine, Rochester, Minnesota. A.J. Halvorsen is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. F.S. McDonald is professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota, and vice president, Graduate Medical Education, American Board of Internal Medicine, Philadelphia, Pennsylvania. D.A. Reed is associate professor of medicine and of medical education, Mayo Clinic College of Medicine, Rochester, Minnesota
| | | | | | | | | | | | | |
Collapse
|
36
|
Gosman A, Mann K, Reid CM, Vedder NB, Janis JE. Implementing Assessment Methods in Plastic Surgery. Plast Reconstr Surg 2016; 137:617e-623e. [DOI: 10.1097/01.prs.0000479968.76438.27] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
37
|
Johnston MJ, Paige JT, Aggarwal R, Stefanidis D, Tsuda S, Khajuria A, Arora S. An overview of research priorities in surgical simulation: what the literature shows has been achieved during the 21st century and what remains. Am J Surg 2016; 211:214-25. [DOI: 10.1016/j.amjsurg.2015.06.014] [Citation(s) in RCA: 36] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2015] [Accepted: 06/26/2015] [Indexed: 10/23/2022]
|
38
|
Pang CL, Patel SB, Pilkington N. Lessons from surgery and anaesthesia: evaluation of non-technical skills in interventional radiology. JRSM Open 2015; 6:2054270415611834. [PMID: 26664733 PMCID: PMC4668915 DOI: 10.1177/2054270415611834] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
In the medical profession, surgery and anaesthesia are leading the way in identifying human errors that negatively affect patient safety. Evidence suggests that the implementation of non-technical skills assessments reduces such errors. Interventional Radiology is a procedural based speciality and therefore may also benefit from formal assessment of non-technical skills. This literature review supports the use of standardised assessment tools used in surgery and anaesthesia. Using the Downing framework of internal validity, the tools demonstrated good internal consistency but a spectrum of inter-rater variability, which can be partially improved with training. At present, a formal Interventional Radiology non-technical skills assessment tool is probably not suitable to be a stand-alone ‘high stakes’ assessment, but may be a useful adjunct to the existing array of workplace-based assessments.
Collapse
Affiliation(s)
- Chun L Pang
- Peninsula Radiology Academy, Plymouth NHS Hospitals Trust, Plymouth, UK
| | - Salil B Patel
- Peninsula College of Medicine and Dentistry, Royal Devon and Exeter Hospital, Exeter, UK
| | | |
Collapse
|
39
|
Steinemann S, Kurosawa G, Wei A, Ho N, Lim E, Suares G, Bhatt A, Berg B. Role confusion and self-assessment in interprofessional trauma teams. Am J Surg 2015; 211:482-8. [PMID: 26801092 DOI: 10.1016/j.amjsurg.2015.11.001] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2015] [Revised: 11/17/2015] [Accepted: 11/22/2015] [Indexed: 01/09/2023]
Abstract
BACKGROUND Trauma care requires coordinating an interprofessional team, with formative feedback on teamwork skills. We hypothesized nurses and surgeons have different perceptions regarding roles during resuscitation; that nurses' teamwork self-assessment differs from experts', and that video debriefing might improve accuracy of self-assessment. METHODS Trauma nurses and surgeons were surveyed regarding resuscitation responsibilities. Subsequently, nurses joined interprofessional teams in simulated trauma resuscitations. After each resuscitation, nurses and teamwork experts independently scored teamwork (T-NOTECHS). After video debriefing, nurses repeated T-NOTECHS self-assessment. RESULTS Nurses and surgeons assumed significantly more responsibility by their own profession for 71% of resuscitation tasks. Nurses' overall T-NOTECHS ratings were slightly higher than experts'. This was evident in all T-NOTECHS subdomains except "leadership," but despite statistical significance the difference was small and clinically irrelevant. Video debriefing did not improve the accuracy of self-assessment. CONCLUSIONS Nurses and physicians demonstrated discordant perceptions of responsibilities. Nurses' self-assessment of teamwork was statistically, but not clinically significantly, higher than experts' in all domains except physician leadership.
Collapse
Affiliation(s)
- Susan Steinemann
- Department of Surgery, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI; Department of Biostatistics and Quantitative Health Sciences, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI.
| | - Gene Kurosawa
- Department of Surgery, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI
| | - Alexander Wei
- Department of Surgery, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI
| | - Nina Ho
- Department of Surgery, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI
| | - Eunjung Lim
- Department of Biostatistics and Quantitative Health Sciences, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI
| | - Gregory Suares
- Department of Surgery, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI; The Queen's Medical Center, Punchbowl Street, Honolulu, HI
| | - Ajay Bhatt
- Department of Surgery, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI; The Queen's Medical Center, Punchbowl Street, Honolulu, HI
| | - Benjamin Berg
- Department of Medicine, John A. Burns School of Medicine, University of Hawaii, Honolulu, HI
| |
Collapse
|
40
|
Bezemer J, Korkiakangas T, Weldon SM, Kress G, Kneebone R. Unsettled teamwork: communication and learning in the operating theatres of an urban hospital. J Adv Nurs 2015; 72:361-72. [DOI: 10.1111/jan.12835] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/10/2015] [Indexed: 11/29/2022]
|
41
|
Lindeman BM, Sacks BC, Lipsett PA. Graduating Students' and Surgery Program Directors' Views of the Association of American Medical Colleges Core Entrustable Professional Activities for Entering Residency: Where are the Gaps? JOURNAL OF SURGICAL EDUCATION 2015; 72:e184-92. [PMID: 26276302 DOI: 10.1016/j.jsurg.2015.07.005] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/11/2015] [Revised: 06/23/2015] [Accepted: 07/08/2015] [Indexed: 05/26/2023]
Abstract
OBJECTIVE Residency program directors have increasingly expressed concern about the preparedness of some medical school graduates for residency training. The Association of American Medical Colleges recently defined 13 core entrustable professional activities (EPAs) for entering residency that residents should be able to perform without direct supervision on the first day of training. It is not known how students' perception of their competency with these activities compares with that of surgery program directors'. DESIGN Cross-sectional survey. SETTING All surgery training programs in the United States. PARTICIPANTS All program directors (PDs) in the Association of Program Directors in Surgery (APDS) database (n = 222) were invited to participate in an electronic survey, and 119 complete responses were received (53.6%). Among the respondents, 83% were men and 35.2% represented community hospital programs. PDs' responses were compared with questions asking students to rate their confidence in performance of each EPA from the Association of American Medical Colleges Graduation Questionnaire (95% response). RESULTS PDs rated their confidence in residents' performance without direct supervision for every EPA significantly lower when compared with the rating by graduating students. Although PDs' ratings continued to be lower than students' ratings, PDs from academic programs (those associated with a medical school) gave higher ratings than those from community programs. PDs generally ranked all 13 EPAs as important to being a trustworthy physician. PDs from programs without preliminary residents gave higher ratings for confidence with EPA performance as compared with PDs with preliminary residents. Among PDs with preliminary residents, there were equal numbers of those who agreed and those who disagreed that there are no identifiable differences between categorical and preliminary residents (42.7% and 41.8%, respectively). CONCLUSIONS A large gap exists between confidence in performance of the 13 core EPAs for entering residency without direct supervision for graduating medical students and surgery program directors. Both the groups identified several key areas for improvement that may be addressed by medical school curricular interventions or expanding surgical boot camps in hopes to improve resident performance and patient safety.
Collapse
Affiliation(s)
- Brenessa M Lindeman
- Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland.
| | - Bethany C Sacks
- Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Pamela A Lipsett
- Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| |
Collapse
|
42
|
van der Poel H, Brinkman W, van Cleynenbreugel B, Kallidonis P, Stolzenburg JU, Liatsikos E, Ahmed K, Brunckhorst O, Khan MS, Do M, Ganzer R, Murphy DG, Van Rij S, Dundee PE, Dasgupta P. Training in minimally invasive surgery in urology: European Association of Urology/International Consultation of Urological Diseases consultation. BJU Int 2015; 117:515-30. [DOI: 10.1111/bju.13320] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Affiliation(s)
- Henk van der Poel
- Department of Urology; Netherlands Cancer Institute; Amsterdam The Netherlands
| | - Willem Brinkman
- Department of Urology; Erasmus Medical Centre; Rotterdam The Netherlands
| | | | - Panagiotis Kallidonis
- Department of Urology; University of Leipzig; Leipzig Germany
- Department of Urology; University of Patras; Patras Greece
| | | | | | - Kamran Ahmed
- MRC Centre for Transplantation; NIHR Biomedical Research Centre; King's College London; London UK
- Department of Urology; Guy's and St. Thomas’ NHS Foundation Trust; King's Health Partners; London UK
| | - Oliver Brunckhorst
- MRC Centre for Transplantation; NIHR Biomedical Research Centre; King's College London; London UK
- Department of Urology; Guy's and St. Thomas’ NHS Foundation Trust; King's Health Partners; London UK
| | - Mohammed Shamim Khan
- MRC Centre for Transplantation; NIHR Biomedical Research Centre; King's College London; London UK
- Department of Urology; Guy's and St. Thomas’ NHS Foundation Trust; King's Health Partners; London UK
| | - Minh Do
- Department of Urology; University of Leipzig; Leipzig Germany
| | - Roman Ganzer
- Department of Urology; University of Leipzig; Leipzig Germany
| | - Declan G. Murphy
- Division of Cancer Surgery; Peter MacCallum Cancer Centre; University of Melbourne; Melbourne Vic. Australia
- Epworth Prostate Centre; Epworth Healthcare; Richmond SA Australia
- Royal Melbourne Hospital; Melbourne Vic. Australia
| | - Simon Van Rij
- Division of Cancer Surgery; Peter MacCallum Cancer Centre; University of Melbourne; Melbourne Vic. Australia
| | - Philip E. Dundee
- Epworth Prostate Centre; Epworth Healthcare; Richmond SA Australia
- Royal Melbourne Hospital; Melbourne Vic. Australia
| | - Prokar Dasgupta
- MRC Centre for Transplantation; NIHR Biomedical Research Centre; King's College London; London UK
- Department of Urology; Guy's and St. Thomas’ NHS Foundation Trust; King's Health Partners; London UK
| |
Collapse
|
43
|
Vincent C, Wearden A, French DP. Making health care safer: What is the contribution of health psychology? Br J Health Psychol 2015; 20:681-7. [PMID: 26440293 DOI: 10.1111/bjhp.12166] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2015] [Revised: 08/21/2015] [Indexed: 01/01/2023]
Abstract
While health care brings great benefits, all treatments, and many investigations, carry some risk. As patients, we should be told of the risks of specific treatments but we are also at risk from failings in the health care system itself. We suggest that, while there are many examples of individual health psychologists who have made important contributions, this has not yet translated into a broader disciplinary engagement. Health psychologists have devoted much more attention to patients and devoted much less attention to the potentially huge impact of studying and intervening with staff, clinical practice, and organizations. We believe that there are considerable opportunities for health psychology to engage more closely with patient safety and, more importantly, that this would be of great benefit to both patients and staff. Statement of contribution What is already known on this subject? While health care brings great benefits, all treatments, and many investigations, carry some risk. Patients are also at risk from failings in the health care system itself. Studies using review of medical records in many countries have found that between 8% and 12% of patients in hospital suffer an unintended harm due to health care. What does this study add? There are many examples of individual psychologists who have made important contributions, but this has not yet translated into a broader disciplinary engagement. There are considerable opportunities for health psychology to engage more closely with patient safety. These include health behaviour change, teamwork, communication after medical error, diagnosis and decision making, organisational culture, and improving compliance with rules and standards. Psychologists providing a clinical service to specialist services in any area could expand their remit from supporting patients to a more general support and engagement with safety and quality initiatives. Health psychologists have models to understand the behaviour of people, and recent developments in changing behaviour should be applicable to health professionals in addition to their patients.
Collapse
Affiliation(s)
- Charles Vincent
- Department of Experimental Psychology, University of Oxford, UK
| | - Alison Wearden
- Manchester Centre for Health Psychology, School of Psychological Sciences, University of Manchester, UK
| | - David P French
- School of Psychological Sciences, University of Manchester, UK
| |
Collapse
|
44
|
Wingo MT, Havyer RDA, Comfere NI, Nelson DR, Reed DA. Interprofessional collaboration milestones: advocating for common assessment criteria in graduate medical education. BMC MEDICAL EDUCATION 2015; 15:149. [PMID: 26369325 PMCID: PMC4570478 DOI: 10.1186/s12909-015-0432-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2015] [Accepted: 09/04/2015] [Indexed: 05/14/2023]
Abstract
BACKGROUND Milestone-based assessments of resident physicians inform critical decisions regarding resident competence and advancement. Thus, it is essential that milestone evaluations are based upon strong validity evidence and that consistent evaluation criteria are used across residency programs. A common approach to assessment of interprofessional collaboration milestones is particularly important since standardized measures of individual resident competence in interprofessional collaboration have not been established. DISCUSSION We propose that assessments of interprofessional collaboration in graduate medical education meet common criteria, namely, these assessments should: 1) measure competency of an individual resident, 2) occur in the context of an interprofessional team, 3) be ascertained via direct observation of the resident, 4) be performed in a real-world clinical practice setting (such as a hospital ward, outpatient clinic, or operating room). We present the evidence-based rationale for these criteria and cite examples of published assessment instruments that fulfill one or more of the criteria, however further research is needed to ensure fidelity of assessments. The proposed criteria may assist residency educators as they endeavor to provide robust and consistent assessments of interprofessional collaboration milestones.
Collapse
Affiliation(s)
- Majken T Wingo
- Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| | - Rachel D A Havyer
- Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| | - Nneka I Comfere
- Departments of Dermatology and Laboratory Medicine & Pathology, Mayo Clinic, Rochester, MN, USA.
| | - Darlene R Nelson
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| | - Darcy A Reed
- Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
45
|
Teamwork Assessment Tools in Modern Surgical Practice: A Systematic Review. Surg Res Pract 2015; 2015:494827. [PMID: 26425732 PMCID: PMC4573989 DOI: 10.1155/2015/494827] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2015] [Revised: 08/21/2015] [Accepted: 08/23/2015] [Indexed: 12/22/2022] Open
Abstract
Introduction. Deficiencies in teamwork skills have been shown to contribute to the occurrence of adverse events during surgery. Consequently, several teamwork assessment tools have been developed to evaluate trainee nontechnical performance. This paper aims to provide an overview of these instruments and review the validity of each tool. Furthermore, the present paper aims to review the deficiencies surrounding training and propose several recommendations to address these issues. Methods. A systematic literature search was conducted to identify teamwork assessment tools using MEDLINE (1946 to August 2015), EMBASE (1974 to August 2015), and PsycINFO (1806 to August 2015) databases. Results. Eight assessment tools which encompass aspects of teamwork were identified. The Nontechnical Skills for Surgeons (NOTSS) assessment was found to possess the highest level of validity from a variety of sources; reliability and acceptability have also been established for this tool. Conclusions. Deficits in current surgical training pathways have prompted several recommendations to meet the evolving requirements of surgeons. Recommendations from the current paper include integration of teamwork training and assessment into medical school curricula, standardised formal training of assessors to ensure accurate evaluation of nontechnical skill acquisition, and integration of concurrent technical and nontechnical skills training throughout training.
Collapse
|
46
|
Validity and reliability assessment of detailed scoring checklists for use during perioperative emergency simulation training. Simul Healthc 2015; 9:295-303. [PMID: 25188486 DOI: 10.1097/sih.0000000000000048] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
INTRODUCTION Few valid and reliable grading checklists have been published for the evaluation of performance during simulated high-stakes perioperative event management. As such, the purposes of this study were to construct valid scoring checklists for a variety of perioperative emergencies and to determine the reliability of scores produced by these checklists during continuous video review. METHODS A group of anesthesiologists, intensivists, and educators created a set of simulation grading checklists for the assessment of the following scenarios: severe anaphylaxis, cerebrovascular accident, hyperkalemic arrest, malignant hyperthermia, and acute coronary syndrome. Checklist items were coded as critical or noncritical. Nonexpert raters evaluated 10 simulation videos in a random order, with each video being graded 4 times. A group of faculty experts also graded the videos to create a reference standard to which nonexpert ratings were compared. P < 0.05 was considered significant. RESULTS Team leaders in the simulation videos were scored by the expert panel as having performed 56.5% of all items on the checklist (range, 43.8%-84.0%), and 67.2% of the critical items (range, 30.0%-100%). Nonexpert raters agreed with the expert assessment 89.6% of the time (95% confidence interval, 87.2%-91.6%). No learning curve development was found with repetitive video assessment or checklist use. The κ values comparing nonexpert rater assessments to the reference standard averaged 0.76 (95% confidence interval, 0.71-0.81). CONCLUSIONS The findings indicate that the grading checklists described are valid, are reliable, and could be used in perioperative crisis management assessment.
Collapse
|
47
|
Kiesewetter J, Fischer MR. The Teamwork Assessment Scale: A Novel Instrument to Assess Quality of Undergraduate Medical Students' Teamwork Using the Example of Simulation-based Ward-Rounds. GMS ZEITSCHRIFT FUR MEDIZINISCHE AUSBILDUNG 2015; 32:Doc19. [PMID: 26038684 PMCID: PMC4446650 DOI: 10.3205/zma000961] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2014] [Revised: 02/20/2015] [Accepted: 03/13/2015] [Indexed: 11/30/2022]
Abstract
Background: Simulation-based teamwork trainings are considered a powerful training method to advance teamwork, which becomes more relevant in medical education. The measurement of teamwork is of high importance and several instruments have been developed for various medical domains to meet this need. To our knowledge, no theoretically-based and easy-to-use measurement instrument has been published nor developed specifically for simulation-based teamwork trainings of medical students. Internist ward-rounds function as an important example of teamwork in medicine. Purposes: The purpose of this study was to provide a validated, theoretically-based instrument that is easy-to-use. Furthermore, this study aimed to identify if and when rater scores relate to performance. Methods: Based on a theoretical framework for teamwork behaviour, items regarding four teamwork components (Team Coordination, Team Cooperation, Information Exchange, Team Adjustment Behaviours) were developed. In study one, three ward-round scenarios, simulated by 69 students, were videotaped and rated independently by four trained raters. The instrument was tested for the embedded psychometric properties and factorial structure. In study two, the instrument was tested for construct validity with an external criterion with a second set of 100 students and four raters. Results: In study one, the factorial structure matched the theoretical components but was unable to separate Information Exchange and Team Cooperation. The preliminary version showed adequate psychometric properties (Cronbach’s α=.75). In study two, the instrument showed physician rater scores were more reliable in measurement than those of student raters. Furthermore, a close correlation between the scale and clinical performance as an external criteria was shown (r=.64) and the sufficient psychometric properties were replicated (Cronbach’s α=.78). Conclusions: The validation allows for use of the simulated teamwork assessment scale in undergraduate medical ward-round trainings to reliably measure teamwork by physicians. Further studies are needed to verify the applicability of the instrument.
Collapse
Affiliation(s)
- Jan Kiesewetter
- Klinikum der LMU München, Institut für Didaktik und Ausbildungsforschung in der Medizin, München, Germany ; Ludwig-Maximilians-Universtiy Munich, Munich Center of the Learning Sciences (MCLS), Munich, Germany
| | - Martin R Fischer
- Klinikum der LMU München, Institut für Didaktik und Ausbildungsforschung in der Medizin, München, Germany
| |
Collapse
|
48
|
Stefanidis D, Sevdalis N, Paige J, Zevin B, Aggarwal R, Grantcharov T, Jones DB. Simulation in Surgery. Ann Surg 2015; 261:846-53. [DOI: 10.1097/sla.0000000000000826] [Citation(s) in RCA: 142] [Impact Index Per Article: 15.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
49
|
Ahmed K, Khan R, Mottrie A, Lovegrove C, Abaza R, Ahlawat R, Ahlering T, Ahlgren G, Artibani W, Barret E, Cathelineau X, Challacombe B, Coloby P, Khan MS, Hubert J, Michel MS, Montorsi F, Murphy D, Palou J, Patel V, Piechaud PT, Van Poppel H, Rischmann P, Sanchez-Salas R, Siemer S, Stoeckle M, Stolzenburg JU, Terrier JE, Thüroff JW, Vaessen C, Van Der Poel HG, Van Cleynenbreugel B, Volpe A, Wagner C, Wiklund P, Wilson T, Wirth M, Witt J, Dasgupta P. Development of a standardised training curriculum for robotic surgery: a consensus statement from an international multidisciplinary group of experts. BJU Int 2015; 116:93-101. [PMID: 25359658 DOI: 10.1111/bju.12974] [Citation(s) in RCA: 103] [Impact Index Per Article: 11.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
OBJECTIVES To explore the views of experts about the development and validation of a robotic surgery training curriculum, and how this should be implemented. MATERIALS AND METHODS An international expert panel was invited to a structured session for discussion. The study was of a mixed design, including qualitative and quantitative components based on focus group interviews during the European Association of Urology (EAU) Robotic Urology Section (ERUS) (2012), EAU (2013) and ERUS (2013) meetings. After introduction to the aims, principles and current status of the curriculum development, group responses were elicited. After content analysis of recorded interviews generated themes were discussed at the second meeting, where consensus was achieved on each theme. This discussion also underwent content analysis, and was used to draft a curriculum proposal. At the third meeting, a quantitative questionnaire about this curriculum was disseminated to attendees to assess the level of agreement with the key points. RESULTS In all, 150 min (19 pages) of the focus group discussion was transcribed (21 316 words). Themes were agreed by two raters (median agreement κ 0.89) and they included: need for a training curriculum (inter-rater agreement κ 0.85); identification of learning needs (κ 0.83); development of the curriculum contents (κ 0.81); an overview of available curricula (κ 0.79); settings for robotic surgery training ((κ 0.89); assessment and training of trainers (κ 0.92); requirements for certification and patient safety (κ 0.83); and need for a universally standardised curriculum (κ 0.78). A training curriculum was proposed based on the above discussions. CONCLUSION This group proposes a multi-step curriculum for robotic training. Studies are in process to validate the effectiveness of the curriculum and to assess transfer of skills to the operating room.
Collapse
Affiliation(s)
- Kamran Ahmed
- Department of Urology, Medical Research Council (MRC) Centre for Transplantation, King's College London, Guy's Hospital, London, UK
| | - Reenam Khan
- Department of Urology, Medical Research Council (MRC) Centre for Transplantation, King's College London, Guy's Hospital, London, UK
| | - Alexandre Mottrie
- Department of Urology, OLV Vattikuti Robotic Surgery Institute, OLV Hospital, Aalst, Belgium
| | - Catherine Lovegrove
- Department of Urology, Medical Research Council (MRC) Centre for Transplantation, King's College London, Guy's Hospital, London, UK
| | - Ronny Abaza
- Department of Urology, The Ohio State University Comprehensive Cancer Center, Arthur G James Cancer Hospital & Richard J Solove Research Institute, Columbus, OH, USA
| | | | - Thomas Ahlering
- Department of Urology, University of California, Irvine, Orange, CA, USA
| | - Goran Ahlgren
- Department of Urology, Lund University Hospital, Lund, Sweden
| | | | - Eric Barret
- Department of Urology, Institut Mutualiste Montsouris, Paris, France
| | | | - Ben Challacombe
- Department of Urology, Medical Research Council (MRC) Centre for Transplantation, King's College London, Guy's Hospital, London, UK
| | - Patrick Coloby
- Service d'Urologie, Centre Hospitalier René-Dubos, Cergy-Pontoise, France
| | - Muhammad S Khan
- Department of Urology, Medical Research Council (MRC) Centre for Transplantation, King's College London, Guy's Hospital, London, UK
| | - Jacques Hubert
- Service d'Urologie, CHRU Nancy, Vandoeeuvre-lès-Nancy, France
| | | | | | - Declan Murphy
- Peter MacCallum Cancer Centre and the Royal Melbourne Hospital, Melbourne, Vic., Australia
| | - Joan Palou
- Department of Urology, Fundació Puigvert, Universitat Autònoma de Barcelona, Barcelona, Spain
| | - Vipul Patel
- Global Robotics Institute, Florida Hospital Celebration Health, Celebration, FL, USA
| | | | | | | | | | - Stefan Siemer
- Klinik für Urologie und Kinderurologie, Universitätsklinikum des Saarlandes, Homburg/Saar, Germany
| | - Michael Stoeckle
- Klinik für Urologie und Kinderurologie, Universitätsklinikum des Saarlandes, Homburg/Saar, Germany
| | | | | | | | - Christophe Vaessen
- Service D'Urologie et de Transplantation Réno-Pancréatique, Hôpital Pitié-Salpêtrière, Paris, France
| | - Henk G Van Der Poel
- Department Urology, Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Alessandro Volpe
- Department of Urology, OLV Vattikuti Robotic Surgery Institute, OLV Hospital, Aalst, Belgium.,University of Eastern Piedmont, Novara, Italy
| | | | - Peter Wiklund
- Department of Oncology and Pathology, Karolinska Institute, Stockholm, Sweden
| | | | - Manfred Wirth
- Department of Urology, University Hospital Carl Gustav Carus, Technical University of Dresden, Dresden, Germany
| | - Jörn Witt
- Department of Oncology and Pathology, Karolinska Institute, Stockholm, Sweden
| | - Prokar Dasgupta
- Department of Urology, Medical Research Council (MRC) Centre for Transplantation, King's College London, Guy's Hospital, London, UK
| |
Collapse
|
50
|
Towards the next frontier for simulation-based training: full-hospital simulation across the entire patient pathway. Ann Surg 2015; 260:252-8. [PMID: 24263325 DOI: 10.1097/sla.0000000000000305] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE To evaluate the efficacy of an entire hospital simulation in imparting skills to expert healthcare providers, encompassing both retention and transfer to clinical practice. BACKGROUND Studies demonstrating the effectiveness of simulation do not concentrate upon expert multidisciplinary teams. Moreover, their focus is confined to a single clinical setting, thereby not considering the complex interactions across multiple hospital departments. METHODS A total of 288 participants (Attending surgeons, anesthesiologists, physicians, and nurses) completed this largest simulation study to date, set in the UK Defence Medical Services' Hospital Simulator and the conflict zone in Afghanistan. The simulator termed "Hospital Exercise" (HOSPEX) is a fully immersive live-in simulation experience that covers the entire environment of a military hospital with all departments. Participants undertook a 3-day training program within HOSPEX before deployment to war zones. Primary outcome measures were assessed with IMPAcT (the Imperial Military Personnel Assessment Tool). IMPAcT measures crisis management, trauma care, hospital environment, operational readiness, and transfer of skills to civilian practice. Reliability, skills learning, and retention in the conflict zone were assessed statistically. RESULTS Reliability in skills assessment was excellent (Cronbach α: nontechnical skills = 0.87-0.94; environment/patient skills = 0.83-0.95). Pre/post-HOSPEX comparisons revealed significant improvements in decision making (M = 4.98, SD = 1.20 to M = 5.39, SD = 0.91; P = 0.03), situational awareness (M = 5.44, SD = 1.04 to M = 5.74, SD = 0.92; P = 0.01), trauma care (M = 5.53, SD = 1.23 to M = 5.85, SD = 1.09; P = 0.05), and knowledge of hospital environment (M = 5.19, SD = 1.17 to M = 5.42, SD = 0.97; P = 0.04). No skills decayed over time when assessed several months later in the real conflict zone. All skills transferred to civilian clinical practice. CONCLUSIONS This is the first study to describe the value of a full-hospital simulation across the entire patient pathway. Such macrosimulations may be the way forward for integrating the complex training needs of expert clinicians and testing organizational "fitness for purpose" of entire hospitals.
Collapse
|