Translation and psychometric validation of the Thai version of TeamSTEPPS® team performance observation tool

In healthcare, effective communication and teamwork are pivotal in reducing medical errors. Integrating team training into health professions education is crucial. Accurate measurement of team performance during training requires reliable assessment tools. The TeamSTEPPS® Team Performance Observation Tool (TPOT), recently updated by the Agency for Healthcare Research and Quality, serves this purpose. However, it had not been translated and validated for use in Thailand. We aimed to translate and assess the psychometric properties of the Thai version of TPOT. Employing a back-translation process, TPOT was adapted to the Thai context. The resulting Thai TPOT instrument was administered to 518 healthcare professionals who had undergone TeamSTEPPS® training. Participants were asked to evaluate two prerecorded, 7-minute simulated team emergency scenarios using the Thai TPOT instrument. Results exhibited high internal consistency (Cronbach's alpha = .96) and inter-rater reliability (ICC = .98). Confirmatory factor analysis affirmed the construct validity of the Thai TPOT. These findings establish the Thai TPOT as useful for evaluating teamwork within healthcare teams.

Development of a Tool to Assess Surgical Resident Competence On-Call: The Western University Call Assessment Tool (WUCAT)

BACKGROUND

A central tenet of competency-based medical education is the formative assessment of trainees. There are currently no assessments designed to examine resident competence on-call, despite the on-call period being a significant component of residency, characterized by less direct supervision compared to daytime. The purpose of this study was to design a formative on-call assessment tool and collect valid evidence on its application.

METHODS

Nominal group technique was used to identify critical elements of surgical resident competence on-call to inform tool development. The tool was piloted over six months in the Division of Plastic & Reconstructive Surgery at our institution. Quantitative and qualitative evidence was collected to examine tool validity.

RESULTS

A ten-item tool was developed based on the consensus group results. Sixty-three assessments were completed by seven staff members on ten residents during the pilot. The tool had a reliability coefficient of 0.67 based on a generalizability study and internal item consistency was 0.92. Scores were significantly associated with years of training. We found the tool improved the quantity and structure of feedback given and that the tool was considered feasible and acceptable by both residents and staff members.

CONCLUSIONS

The Western University Call Assessment Tool (WUCAT) has multiple sources of evidence supporting its use in assessing resident competence on-call.

In situ simulation-based team training and its significance for transfer of learning to clinical practice-A qualitative focus group interview study of anaesthesia personnel

BACKGROUND

Anaesthesia personnel are an integral part of an interprofessional operating room-team; hence, team-based training in non-technical skills (NTS) are important in preventing adverse events. Quite a few studies have been done on interprofessional in situ simulation-based team training (SBTT). However, research on anaesthesia personnel's experiences and the significance for transfer of learning to clinical practice is limited. The aim of this study is to explore anaesthesia personnel's experience from interprofessional in situ SBTT in NTS and its significance for transfer of learning to clinical practice.

METHODS

Follow-up focus group interviews with anaesthesia personnel, who had taken part in interprofessional in situ SBTT were conducted. A qualitative inductive content analysis was performed.

RESULTS

Anaesthesia personnel experienced that interprofessional in situ SBTT motivated transfer of learning and provided the opportunity to be aware of own practice regarding NTS and teamwork. One main category, 'interprofessional in situ SBTT as a contributor to enhance anaesthesia practice' and three generic categories, 'interprofessional in situ SBTT motivates learning and improves NTS', 'realism in SBTT is important for learning outcome', and 'SBTT increases the awareness of teamwork' illustrated their experiences.

CONCLUSIONS

Participants in the interprofessional in situ SBTT gained experiences in coping with emotions and demanding situations, which could be significant for transfer of learning essential for clinical practice. Herein communication and decision-making were highlighted as important learning objectives. Furthermore, participants emphasized the importance of realism and fidelity and debriefing in the learning design.

A tool to assess nontechnical skills of perfusionists in the cardiac operating room

OBJECTIVES

This study aimed to develop the Perfusionists' Intraoperative Non-Technical Skills tool, specifically to the perfusionists' context, and test its inter-rater reliability.

METHODS

An expert panel was convened to review existing surgical nontechnical skills taxonomies and develop the Perfusionists' Intraoperative Non-Technical Skills tool. During a workshop held at a national meeting, perfusionists completed the Perfusionists' Intraoperative Non-Technical Skills ratings after watching 4 videos displaying simulated cardiac operations. Two videos showed "good performance," and 2 videos showed "poor performance." Inter-rater reliability analysis was performed and intraclass correlation coefficient was reported.

RESULTS

The final version of the Perfusionists' Intraoperative Non-Technical Skills taxonomy contains 4 behavioral categories (decision making, situation awareness, task management and leadership, teamwork and communication) with 4 behavioral elements each. Categories and elements are rated using an 8-point Likert scale ranging from 0.5 to 4.0. A total of 60 perfusionist raters were included and the comparison between rating distribution on "poor performance" and "good performance" videos yielded a statistically significant difference between groups, with a P value less than .001. A similar difference was found in all behavioral categories and elements. Reliability analysis showed moderate inter-rater reliability across overall ratings (intraclass correlation coefficient, 0.735; 95% confidence interval, 0.674-0.796; P < .001). Similar inter-rater reliability was found when raters were stratified by experience level.

CONCLUSIONS

The Perfusionists' Intraoperative Non-Technical Skills tool presented moderate inter-rater reliability among perfusionists with varied levels of experience. This tool can be used to train and assess perfusionists in relevant nontechnical skills, with the potential to enhance safety and improve surgical outcomes.

Cross-cultural adaptation and validation of the Spanish version of the Anesthetists' Non-Technical Skills (ANTS) assessment tool

STUDY OBJECTIVE

Establish the transcultural validity of Anesthetists Non-Technical Skills (ANTS) in a Spanish-speaking country.

DESIGN

Prospective cohort.

SETTING

Clinical simulation center.

SUBJECTS

Forty-two Anesthesia PY2 and PY3 residents participated in the study.

INTERVENTIONS

Four clinical scenarios simulating anesthesia crises were assessed with a Spanish version of ANTS. Every simulated scenario was run twice with a time span of 3 to 4 months between them.

MEASUREMENTS

Two anesthesiologists independently assessed all simulated sessions using ANTS. The ANTS indicators of construct validity were obtained by confirmatory factor analysis. Various goodness-of-fit indices of the factorial model were calculated: Comparative Fit Index (CFI); Tucker-Lewis Adjustment Index (TLI) and Root Mean Square Error of Approximation (RMSEA). The standardized factor loadings and the determination coefficient (R2) was also estimated.

MAIN RESULTS

A total of 212 clinical scenarios were analyzed. The specified factorial model had the same grouping of elements in four domains as the original version of ANTS. The CFI index and the TLI were 0.99 and the RMSEA reached 0.07 (95% CI 0.06-0.08). All the standardized factor loadings were found to be >0.4. Also, the elements obtained an R2 value that fluctuated between 0.54 and 0.92.

CONCLUSIONS

The Spanish version of ANTS is a valid, reliable and a useful tool to assess non-technical skills in Spanish-speaking countries. The applicability of the instrument was comparable to the original setting. The high reliability of ANTS in our setting allows us to propose its use not just in an educational and research setting; it can be used as an assessment tool of non-technical skills.

Instruments to evaluate non-technical skills during high fidelity simulation: A systematic review

INTRODUCTION

High Fidelity Simulations (HFS) are increasingly used to develop Non-Technical Skills (NTS) in healthcare providers, medical and nursing students. Instruments to measure NTS are needed to evaluate the healthcare providers' (HCPs) performance during HFS. The aim of this systematic review is to describe the domains, items, characteristics and psychometric properties of instruments devised to evaluate the NTS of HCPs during HFS.

METHODS

A systematic review of the literature was performed according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). Studies were retrieved from PubMed, Cinahl, Web of Science, Cochrane Library, ProQuest and PubPsych. Studies evaluating the measurement properties of instruments used to assess NTS during HFS training were included. Pairs of independent reviewers determined the eligibility, extracted and evaluated the data. Risk of bias and appraisal of the methodological quality of the studies was assessed using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist, and the quality of the evidence with the Grading of Recommendations, Assessment, Development and Evaluation (GRADE).

RESULTS

A total of 3,953 articles were screened. A total of 110 reports were assessed for eligibility and 26 studies were included. Studies were conducted in Europe/United Kingdom (n = 13; 50%), North America/Australia (n = 12; 46%) and Thailand (n = 1; 4%). The NTS instruments reported in this review included from 1 to 14 domains (median of 4, Q1 = 3.75, Q3 = 5) and from 3 to 63 items (median of 15, Q1 = 10, Q3 = 19.75). Out of 19 NTS assessment instruments for HFS, the Team Emergency Assessment Measure (TEAM) can be recommended for use to assess NTS. All the other instruments require further research to assess their quality in order to be recommended for use during HFS training. Eight NTS instruments had a positive overall rating of their content validity with at least a moderate quality of evidence.

CONCLUSION

Among a large variety of published instruments, TEAM can be recommended for use to assess NTS during HFS. Evidence is still limited on essential aspects of validity and reliability of all the other NTS instruments included in this review. Further research is warranted to establish their performance in order to be reliably used for HFS.

Is it valid to assess an individual's performance in team training simulation when the supporting team are confederates? A controlled and randomized clinical trial

BACKGROUND

During simulation training, the confederate is a member of the pedagogical team. Its role is to facilitate the interaction between participants and the environment, and is thought to increase realism and immersion. Its influence on participants' performance in full-scale simulation remains however unknown. The purpose of this study was to observe the effect of the presence of confederates on the participants' performance during full-scale simulation of crisis medical situations.

METHODS

This was a prospective, randomized study comparing 2 parallel groups. Participants were emergency medicine residents engaging in a simulation session, with or without confederates. Participants were then evaluated on their Crisis Resource Management performance (CRM). The overall performance score on the Ottawa Global Rating Scale was assessed as primary outcome and the 5 non-technical CRM skills as secondary outcomes.

RESULTS

A total of 63 simulation sessions, including 63 residents, were included for statistical analysis (n = 32 for Control group and 31 for Confederate group). The mean Overall Performance score was 3.9 ± 0.8 in the Control group and 4.0 ± 1.1 in the Confederate group, 95% confidence interval of the difference [-0.6; 0.4], p = 0.60. No significant differences between the two groups were observed on each CRM items (leadership, situational awareness, communication, problem solving, resource utilization) CONCLUSION: In this randomized and controlled study, the presence of confederates during full-scale simulated practice of crisis medical situations does not seem to influence the CRM skills performance of Emergency medicine residents.

TRIAL REGISTRATION

This study does not need to be registered on Clintrial as it does not report a health care intervention on human participants.

Personality in anaesthesiologists, a systematic review of the literature

BACKGROUND

As a central part of their job, anaesthesiologists often have to perform demanding tasks under high-stakes conditions. Yet, some anaesthesiologists seem better able to deal with the demands of the profession than others.

OBJECTIVES

This review aims to answer the following questions. What are the necessary or desirable qualities of an anaesthesiologist? Which personality traits or characteristics have been found in anaesthesiologists? How does personality relate to job performance and work stress among anaesthesiologists?

DESIGN

Systematic review of studies that examined anaesthesiologists' personality or personality characteristics. We performed our synthesis in terms of the five-factor model of personality.

DATA SOURCES

The search was conducted in the PubMed, EMBASE and Web of Science databases. Literature was included until December 2020.

ELIGIBILITY CRITERIA

We included qualitative and quantitative studies that examined anaesthesiologists' personality; also, we included studies that focused on anaesthesiologists' stress, performance or mental health but only if these topics were examined from the perspective of personality.

RESULTS

We included 6 qualitative and 25 quantitative articles. Synthesis of the qualitative articles revealed two classes of desirable technical and nontechnical personality characteristics. Synthesis of the quantitative articles suggested that anaesthesiologists do not essentially differ from other medical specialists. Moreover, our synthesis revealed several personality traits that predict good performance, low stress and good mental health among anaesthesiologists: lower Neuroticism, higher Extraversion, higher Openness and higher Conscientiousness.

CONCLUSION

Those personality traits that predict performance, stress or mental health in anaesthesiologists, also predict performance, stress or mental health in other high demand/high stakes environments (both medical and nonmedical). The ideal anaesthesiologist would be lower on Neuroticism, higher on Extraversion and higher on Conscientiousness.

Validation in Spanish of the Ottawa scale for non-technical skills in health personnel in crisis situations

INTRODUCTION

Non-technical skills are related to morbi-mortality in medicine; it has been proposed that 46% of fatal outcomes are explained to limitations in non-technical skills and only 5% to technical skills deficiencies, however, there is no validated instrument or scale in spanish that allows its evaluation in the management of medical crisis.

OBJECTIVE

To evaluate the psychometric properties of a Spanish-adapted version of the "Ottawa crisis resource management (CRM) global rating scale (GRS)" in medical staff involved in critical decision-making based in high-fidelity simulation, which could be beneficial to impact patient safety and improve clinical outcomes.

METHODS

Transversal cultural instrument validation and adaptation study, included 91 participants who simulated a medical crisis between 2018 and 2019, and to whom the Spanish version of the CRM-GRS was applied in order to evaluate its psychometric properties.

RESULTS

A cultural adaptation with translation into Spanish of the CRM-GRS was made. Subsequently, the scale was applied to 91 participants. An internal consistency (Cronbach's alpha) greater than 0.9 was found in each dimension. The level of inter-rater reliability, evaluated by the interclass coefficient was 0.59 to 0.69, and test-retest reliability with an interclass coefficient greater than 0.7. The validity of the convergent construct was moderate (interclass coefficient between 0.6 and 0.7 for all domains) and the validity of the divergent construct between 0.4 and 0.5 was found adequate.

CONCLUSION

The translated and adapted Spanish version of the CRM-GRS in crisis had adequate internal consistency, reliability, and construct validity.

Validation in spanish of the Ottawa scale for non-techical skills in health personnel in crisis situations

INTRODUCTION

Non-technical skills are related to morbi-mortality in medicine; it has been proposed that 46% of fatal outcomes are explained to limitations in non-technical skills and only 5% to technical skills deficiencies, however, there is no validated instrument or scale in spanish that allows its evaluation in the management of medical crisis.

OBJECTIVE

To evaluate the psychometric properties of a Spanish-adapted version of the "Ottawa crisis resource management (CRM) global rating scale (GRS)" in medical staff involved in critical decision-making based in high-fidelity simulation, which could be beneficial to impact patient safety and improve clinical outcomes.

METHODS

Transversal cultural instrument validation and adaptation study, included 91 participants who simulated a medical crisis between 2018 and 2019, and to whom the Spanish version of the CRM-GRS was applied in order to evaluate its psychometric properties.

RESULTS

A cultural adaptation with translation into Spanish of the CRM-GRS was made. Subsequently, the scale was applied to 91 participants. An internal consistency (Cronbach's alpha) greater than 0.9 was found in each dimension. The level of inter-rater reliability, evaluated by the interclass coefficient was 0.59 to 0.69, and test-retest reliability with an interclass coefficient greater than 0.7. The validity of the convergent construct was moderate (interclass coefficient between 0.6 and 0.7 for all domains) and the validity of the divergent construct between 0.4 and 0.5 was found adequate.

CONCLUSION

The translated and adapted Spanish version of the CRM-GRS in crisis had adequate internal consistency, reliability, and construct validity.

Cross-cultural adaptation and validation of two crisis resource management scales

BACKGROUND

Simulation training programs in crisis resource management must be evaluated using valid and reliable instruments. We translated into Spanish and linguistically validate The Mayo High Performance Teamwork Scale (MHPTS) and Ottawa Crisis Resource Management Global Rating Scale (Ottawa GRS) non-technical skills assessment instruments.

METHOD

We performed a standardised cross-cultural adaptation process. The psychometric properties of both instruments in their versions adapted to Spanish were subsequently evaluated in a sample of 100 students by using exploratory factor analysis and assessing internal consistency and convergent validity through a total of 94 simulation scenarios in urgent medical situations RESULTS: Our results for the MHPTS showed a one-dimensional structure containing 8 items which explained a total variance of 72.84%; the Ottawa GRS also had a one-dimensional structure, this time with 5 items, which explained a total variance of 91.79%. According to the Cronbach alpha, the internal consistency for the MHPTS was 0.94 (1-8 items) and 0.98 for the Ottawa GRS. In addition, there was a strong correlation between the MHPTS and Ottawa GRS (r = 0.97; p < 0.001).

CONCLUSION

We found strong evidence for the high validity and reliability of the Spanish versions of both these tools when tested in Spanish simulated emergency contexts.

Relationship between mindfulness, stress, and performance in medical students in pediatric emergency simulations

Objectives: Pediatric teams of emergency departments work under extreme stress, which affects high-level cognitive functions, specifically attention and memory. Therefore, the methods of stress management are being sought. Mindfulness as a process of intentionally paying attention to each moment with acceptance of each experience without judgment can potentially contribute to improving the performance of medical teams. Medical simulation is a technique that creates a situation to allow persons to experience a representation of a real event for the purpose of education. It has been shown that emergency medicine simulation may create a high physiological fidelity environment similarly to what is observed in a real emergency room. The aim of our study was to determine whether the technical and non-technical skills of medical students in the course of pediatric high fidelity simulations are related to their mindfulness and stress. Participants and methods: A total of 166 standardized simulations were conducted among students of medicine in three simulation centers of medical universities, assessing: stress sensation (subjectively and heart rate/blood pressure), technical (checklists) and non-technical skills (Ottawa scale) and mindfulness (five facet mindfulness questionnaire): ClinicalTrials.gov ID: NCT03761355. Results: The perception of stress among students was lower and more motivating if they were more mindful. Mindfulness of students correlated positively with avoiding fixation error. In the consecutive simulations the leaders' non-technical skills improved, although no change was noted in their technical skills. Conclusion: The results of our research indicate that mindfulness influence the non-technical skills and the perception of stress of medical students during pediatric emergency simulations. Further research is needed to show whether mindfulness training leads to improvement in this field.

Eye tracking metrics and leader's behavioral performance during a post-partum hemorrhage high-fidelity simulated scenario

BACKGROUND

The use of eye tracking in the simulated setting can help improve our understanding of what sources of information clinicians are using as they deliver routine patient care. The aim of this simulation study was to observe the differences, if any, between the eye tracking patterns of leaders who performed best in a simulated postpartum hemorrhage (PPH) high-fidelity scenario, in comparison with those who performed worst.

METHODS

Forty anesthesia trainees from the University of Catania Medical School were divided into eight teams, to enact four times the same scenario of a patient with postpartum hemorrhage following vaginal delivery. Trainees who were assigned the leader's role wore the eye tracking glasses during the scenario, and their behavioral skills were evaluated by two observers, who reviewed the video recordings of the scenarios using a standardized checklist. The leader's eye tracking metrics, extracted from 27 selected areas of interest (AOI), were recorded by a Tobii Pro Glasses 50 Hz wearable wireless eye tracker. Team performance was evaluated using a PPH checklist. After completion of the study, the leaders were divided into two groups, based on the scores they had received (High-Performance Leader group, HPL, and Low-Performance Leader group, LPL).

RESULTS

In the HPL group, the duration and number of fixations were greater, and the distribution of gaze was uniformly distributed among the various members of the team as compared with the LPL group (with the exception of the participant who performed the role of the obstetrician). The HPL group also looked both at the patient's face and established eye contact with their team members more often and for longer (P < .05). The team performance (PPH checklist) score was greater in the HPL group (P < .001). The LPL group had more and/or longer fixations of technical areas of interest (P < .05).

CONCLUSIONS

Our findings suggest that the leaders who perform the best distribute their gaze across all members of their team and establish direct eye contact. They also look longer at the patient's face and dwell less on areas that are more relevant to technical skills. In addition, the teams led by these best performing leaders fulfilled their clinical task better. The information provided by the eye behaviors of "better-performing physicians" may lay the foundation for the future development of both the assessment process and the educational tools used in simulation.

TRIAL REGISTRATION

Clinical.Trial.Gov ID n. NCT04395963 .

Simulation-based team training in time-critical clinical presentations in emergency medicine and critical care: a review of the literature

BACKGROUND

The use of simulation-based team training has increased over the past decades. Simulation-based team training within emergency medicine and critical care contexts is best known for its use by trauma teams and teams involved in cardiac arrest. In the domain of emergency medicine, simulation-based team training is also used for other typical time-critical clinical presentations. We aimed to review the existing literature and current state of evidence pertaining to non-technical skills obtained via simulation-based team training in emergency medicine and critical care contexts, excluding trauma and cardiac arrest contexts.

METHODS

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. Before the initiation of the study, the protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO) database. We conducted a systematic literature search of 10 years of publications, up to December 17, 2019, in the following databases: PubMed/MEDLINE, EMBASE, Cochrane Library, and CINAHL. Two authors independently reviewed all the studies and extracted data.

RESULTS

Of the 456 studies screened, 29 trials were subjected to full-text review, and 13 studies were included in the final review. None of the studies was randomized controlled trials, and no studies compared simulation training to different modalities of training. Studies were heterogeneous; they applied simulation-training concepts of different durations and intensities and used different outcome measures for non-technical skills. Two studies reached Kirkpatrick level 3. Out of the remaining 11 studies, nine reached Kirkpatrick level 2, and two reached Kirkpatrick level 1.

CONCLUSIONS

The literature on simulation-based team training in emergency medicine is heterogeneous and sparse, but somewhat supports the hypothesis that simulation-based team training is beneficial to teams' knowledge and attitudes toward non-technical skills (Kirkpatrick level 2). Randomized trials are called for to clarify the effect of simulation compared to other modalities of team training. Future research should focus on the transfer of skills and investigate improvements in patient outcomes (Kirkpatrick level 4).

Validity of the Medi-StuNTS behavioural marker system: assessing the non-technical skills of medical students during immersive simulation

Background

The Medical Students’ Non-Technical Skills (Medi-StuNTS) behavioural marker system (BMS) is the first BMS to be developed specifically for medical students to facilitate training in non-technical skills (NTS) within immersive simulated acute care scenarios. In order to begin implementing the tool in practice, validity evidence must be sought. We aimed to assess the validity of the Medi-StuNTS system with reference to Messick’s contemporary validity framework.

Methods

Two raters marked video-recorded performances of acute care simulation scenarios using the Medi-StuNTS system. Three groups were marked: third-year and fourth-year medical students (novices), final-year medical students (intermediates) and core medical trainees (experts). The scores were used to make assessments of relationships to the variable of clinical experience through expert–novice comparisons, inter-rater reliability, observability, exploratory factor analysis, inter-rater disagreements and differential item functioning.

Results

A significant difference was found between the three groups (p<0.005), with experts scoring significantly better than intermediates (p<0.005) and intermediates scoring significantly better than novices (p=0.001). There was a strong positive correlation between the two raters’ scores (r=0.79), and an inter-rater disagreement of more than one point in less than one-fifth of cases. Across all scenarios, 99.7% of skill categories and 84% of skill elements were observable. Factor analysis demonstrated appropriate grouping of skill elements. Inconsistencies in test performance across learner groups were shown specifically in the skill categories of situation awareness and decision making and prioritisation.

Conclusion

We have demonstrated evidence for several aspects of validity of the Medi-StuNTS system when assessing medical students’ NTS during immersive simulation. We can now begin to introduce this system into simulation-based education to maximise NTS training in this group.

Relationship between Executive Functions, Mindfulness, Stress, and Performance in Pediatric Emergency Simulations

Over the past decade, high-fidelity medical simulation has become an accepted and widely used teaching method in pediatrics. Both simulation and work in the real conditions of emergency departments are accompanied by stress that affects the executive functions of participants. One of the methods for reducing stress among medical students and healthcare professionals is the practice of mindfulness. The aim of this study was to examine whether executive functions, mindfulness, and stress are related to the technical and non-technical skills of medical students participating in medical simulations in pediatrics. The study included 153 final-year medical students. A total of 306 high-fidelity simulations of life-threatening situations involving children were conducted. Results: Stress and the coping mechanism of the participants were correlated to their skills during pediatric simulations. Some components of mindfulness, such as non-judgment and conscious action, were positively related to the skills of medical team leaders. Executive functions correlated with the non-technical skills and mindfulness of the medical students. Conclusions: Stress, mindfulness, and executive functions modeled the behavior and skills of medical students during pediatric simulations of life-threatening events. Further research in this area may prove whether mindfulness training will improve learning outcomes in pediatric emergency medicine.

Nontechnical Competency Framework for Health Professionals in All-Hazard Emergency Environment: A Systematic Review

OBJECTIVES

To summarize characteristics and commonalities of non-technical competency frameworks for health professionals in emergency and disaster.

METHODS

An electronic literature search was conducted in PubMed, MEDLINE, ERIC, Scopus, Cochrane database, and Google Scholar to identify original English-language articles related to development, evaluation or application of the nontechnical competency frameworks. Reviewers assessed identified articles for exclusion/inclusion criteria and abstracted data on study design, framework characteristics, and reliability/validity evidence.

RESULTS

Of the 9627 abstracts screened, 65 frameworks were identified from 94 studies that were eligible for result extraction. Sixty (63.8%) studies concentrated on clinical settings. Common scenarios of the studies were acute critical events in hospitals (44;46.8%) and nonspecified disasters (39;41.5%). Most of the participants (76; 80.9%) were clinical practitioners, and participants in 36 (38.3%) studies were multispecialty. Thirty-three (50.8%) and 42 (64.6%) frameworks had not reported evidence on reliability and validity, respectively. Fourteen of the most commonly involved domains were identified from the frameworks.

CONCLUSIONS

Nontechnical competency frameworks applied to multidisciplinary emergency health professionals are heterogeneous in construct and application. A fundamental framework with standardized terminology for the articulation of competency should be developed and validated so as to be accepted and adapted universally by health professionals in all-hazard emergency environment.

Measuring non-technical skills of anaesthesiologists in the operating room: a systematic review of assessment tools and their measurement properties

BACKGROUND

Non-technical skills, such as communication or leadership, are integral to clinical competence in anaesthesia. There is a need for valid and reliable tools to measure anaesthetists' non-technical performance for both initial and continuing professional development. This systematic review aims to summarise the measurement properties of existing assessment tools to determine which tool is most robust.

METHODS

Embase (via OVID), Medline and Medline in Process (via OVID), and reference lists of included studies and previously published relevant systematic reviews were searched (through August 2017). Quantitative studies investigating the measurement properties of tools used to assess anaesthetists' intraoperative non-technical skills, either in a clinical or simulated environment, were included. Pairs of independent reviewers determined eligibility and extracted data. Risk of bias was assessed using the COSMIN checklist.

RESULTS

The search yielded 978 studies, of which 14 studies describing seven tools met the inclusion criteria. Of these, 12 involved simulated crisis settings only. The measurement properties of the Anaesthetists' Non-Technical Skills (ANTS) tool were most commonly assessed (n=9 studies), with studies of two types of validity (content, concurrent) and two types of reliability (internal consistency, interrater). Most of these studies, however, were at serious risk of bias.

CONCLUSIONS

Though there are seven tools for assessing the non-technical skills of anaesthetists, only ANTS has been extensively investigated with regard to its measurement properties. ANTS appears to have acceptable validity and reliability for assessing non-technical skills of anaesthetists in both simulated and clinical settings. Future research should consider additional clinical contexts and types of measurement properties.