Crew Resource Management Training Effectiveness: A Meta-Analysis and Some Critical Needs

Development and validation of a questionnaire, the REST-Q Fire, to identify teamwork-related resources and stressors in firefighting operations

In the complex work environments of firefighting teams, it is often human error or difficulties in teamwork that lead to dangerous situations. To prevent these, it is essential to know the teamwork-related stressors and resources in firefighting operations. A measurement tool is needed to evaluate these stressors and resources. A successive instrument development process was conducted to identify the relevant teamwork-related stressors and resources in firefighting operations. First, interviews with experienced firefighters, and second, a document analysis were conducted and evaluated to provide an overview of the teamwork-related stressors and resources. Thereupon, a questionnaire, the REST-Q Fire, was developed asking about the experienced frequency and intensity of the identified teamwork-related stressors and resources in firefighting operations. Afterwards, an online study with firefighters was conducted (N = 491). CFAs confirmed the assumed structure of the REST-Q Fire and a positive correlation of the intensity of stressors with stress was shown (r = .19 - .27). Further, the resources were overall more frequently and intensively perceived than the stressors. The most important stressors were 'behavior of leaders' (M (SD)frequency = 2.80 (0.83), M (SD)intensity = 3.59 (1.12)), and 'behavior of team members' (M (SD)frequency = 2.77 (0.75), M (SD)intensity = 3.59 (1.05)). The most important resources, on the other hand, were 'knowledge about skills and behavior of team members' (M (SD)frequency = 3.96 (0.63), M (SD)intensity = 4.24 (0.78)), 'watch out for/ check on each other' (M (SD)frequency = 3.96 (0.70), M (SD)intensity = 4.20 (0.80)), and 'reliability of team members' (M (SD)frequency = 3.96 (0.51), M (SD)intensity = 4.16 (0.73)). As a result, training needs for trainees in the fire service and experienced firefighters were derived.

Effectiveness of nontechnical skills educational interventions in the context of emergencies: A systematic review and meta-analysis

INTRODUCTION

In recent years, the importance of training healthcare professionals in nontechnical skills using effective methodologies has been increasingly recognised as a means of preventing clinical errors in the practice of health care. The aim of this study was to evaluate the effectiveness of educational interventions on nontechnical skills in the emergency medical services and/or critical care unit settings.

METHODS

A systematic search was carried out in the PubMed, SCOPUS, CINAHL, and Web of Science databases according to predetermined inclusion and exclusion criteria. After the initial search, 7952 records were selected after duplicates removed. Finally, a selection of 38 studies was included for quantitative analysis. Separate meta-analyses of standardised mean changes were carried out for each outcome measure assuming a random-effects model. Cochran's Q-statistic and I2 index were applied to verify study heterogeneity. Weighted analyses of variance and meta-regressions were conducted to test the influence of potential moderators and funnel plots using Duval and Tweedie's trim-and-fill method, and Egger's regression test were used to examine publication bias.

RESULTS

All the variables analysed had a significant effect size, with the exception of situational awareness (d+ = -0.448; 95% confidence interval [CI] = -1.034, 0.139). The highest mean effect size was found for knowledge (d+ = -0.925; 95% CI = -1.177, -0.673), followed by the mean effect sizes for global nontechnical skills (d+ = -0.642; 95% CI = -0.849, -0.434), team nontechnical skills (d+ = -0.606; 95% CI = -0.949, -0.262), and leadership nontechnical skills (d+ = -0.571; 95% CI = -0.877, -0.264). Similar mean effect sizes were found for attitude (d+ = -0.406; 95% CI = -0.769, -0.044), self-efficacy (d+ = -0.469; 95% CI = -0.874, -0.064), and communication nontechnical skills (d+ = -0.458; 95% CI = -0.818, -0.099). Large heterogeneity among the standardised mean changes was found in the meta-analyses (I2 > 75% and p < .001), except for self-efficacy where I2 = 58.17%, and there was a nonstatistical result for Cochran's Q. This great variability is also reflected in the forest plots.

DISCUSSION

The use of simulation interventions to train emergency and critical care healthcare professionals in nontechnical skills significantly improves levels of knowledge, attitude, self-efficacy, and nontechnical skills performance.

Human-Autonomy Teaming: A Review and Analysis of the Empirical Literature

OBJECTIVE

We define human-autonomy teaming and offer a synthesis of the existing empirical research on the topic. Specifically, we identify the research environments, dependent variables, themes representing the key findings, and critical future research directions.

BACKGROUND

Whereas a burgeoning literature on high-performance teamwork identifies the factors critical to success, much less is known about how human-autonomy teams (HATs) achieve success. Human-autonomy teamwork involves humans working interdependently toward a common goal along with autonomous agents. Autonomous agents involve a degree of self-government and self-directed behavior (agency), and autonomous agents take on a unique role or set of tasks and work interdependently with human team members to achieve a shared objective.

METHOD

We searched the literature on human-autonomy teaming. To meet our criteria for inclusion, the paper needed to involve empirical research and meet our definition of human-autonomy teaming. We found 76 articles that met our criteria for inclusion.

RESULTS

We report on research environments and we find that the key independent variables involve autonomous agent characteristics, team composition, task characteristics, human individual differences, training, and communication. We identify themes for each of these and discuss the future research needs.

CONCLUSION

There are areas where research findings are clear and consistent, but there are many opportunities for future research. Particularly important will be research that identifies mechanisms linking team input to team output variables.

State of science: models and methods for understanding and enhancing teams and teamwork in complex sociotechnical systems

This state of the science review brings together the disparate literature of effective strategies for enhancing and accelerating team performance. The review evaluates and synthesises models and proposes recommended avenues for future research. The two major models of the Input-Mediator-Output-Input (IMOI) framework and the Big Five dimensions of teamwork were reviewed and both will need significant development for application to future teams comprising non-human agents. Research suggests that a multi-method approach is appropriate for team measurements, such as the integration of methods from self-report, observer ratings, event-based measurement and automated recordings. Simulations are recommended as the most effective team-based training interventions. The impact of new technology and autonomous agents is discussed with respect to the changing nature of teamwork. In particular, whether existing teamwork models and measures are suitable to support the design, operation and evaluation of human-nonhuman teams of the future. Practitioner summary: This review recommends a multi-method approach to the measurement and evaluation of teamwork. Team models will need to be adapted to describe interaction with non-human agents, which is what the future is most likely to hold. The most effective team training interventions use simulation-based approaches.

What Do We Really Know About Crew Resource Management in Healthcare?: An Umbrella Review on Crew Resource Management and Its Effectiveness

OBJECTIVE

The aim of this article was to present an overview of the crew resource management (CRM) literature in healthcare. The first aim was to conduct an umbrella review on CRM literature reviews. The second aim was to conduct a new literature review that aims to address the gaps that were identified through the umbrella review.

METHODS

First, we conducted an umbrella review to identify all reviews that have focused on CRM within the healthcare context. This step resulted in 16 literature reviews. Second, we conducted a comprehensive literature review that resulted in 106 articles.

RESULTS

The 16 literature reviews showed a high level of heterogeneity, which resulted in discussing 3 ambiguities: definition, outcome, and information ambiguity. As a result of these ambiguities, a new comprehensive review of the CRM literature was conducted. This review showed that CRM seems to have a positive effect on outcomes at Kirkpatrick's level 1, 2, and 3. In contrast, whether CRM has a positive effect on level 4 outcomes and how level 4 should be measured remains undetermined. Recommendations on how to implement and embed CRM training into an organization to achieve the desired effects have not been adequately considered.

CONCLUSIONS

The extensive nature of this review demonstrates the popularity of CRM in healthcare, but at the same time, it highlights that research tends to be situated within certain settings, focuses on particular outcomes, and has failed to address the full scope of CRM as a team intervention and a management concept.

John Senders, Human Error, and System Safety

OBJECTIVE

I examine John Senders' work and discuss his influence on the study of error causation,error mitigation, and sociotechnical system safety.

BACKGROUND

John Senders' passing calls for an evaluation of the impact of his work.

METHOD

I review literature and accident investigation findings to discuss themes in Senders' work and potential associations between that work and error causation and system safety.

RESULTS

Senders consistently emphasized empirical rigor and theoretical exploration in his research, with the desire to apply that work to enhance human performance. He has contributed to changing the way error has been viewed, and to developing and implementing programs and techniques to mitigate error. While a causal relationship between Senders' work and safety cannot be established, an association can be drawn between his research and efforts to mitigate error.

CONCLUSION

Because of Senders' work, we have a better understanding of error causation and enhanced ways of mitigating system errors. However, new sources of error, involving advanced systems and operators' knowledge and understanding of their functionalities can, if not addressed, degrade system safety.

APPLICATION

Modifications to advanced automation and operator training are suggested, and research to improve operator expertise in interacting with automated systems proposed.

Safety culture transformation-The impact of training on explicit and implicit safety attitudes

The present paper investigates the changeability of safety culture elements such as explicit and implicit safety attitudes by training. Therefore, three studies with different time frames, training durations, and settings will be presented. In the first study, the short-term attitude change of students from an international environmental sciences study program was measured after safety training in a chemical laboratory. In the second study, the medium-term attitude change was assessed after a Crew Resource Management training for German production workers in the automotive industry. In the third study, the long-term attitude changes were measured after safety ethics training in a sample of German occupational psychology and business students. Different self-report measures were used to evaluate the training effectiveness of explicit safety attitudes. The change of implicit safety attitudes was assessed by Implicit Association Tests. The results of all three studies revealed a significant training effect on the explicit safety attitudes, but not on the implicit ones. Besides the training effect on the explicit attitudes, there was no effect of time frame (short-, medium-, long-term), training duration (2 h, 2 days, 12 weeks), and setting (chemical laboratory, automotive industry, safety ethics study program) on the attitude change. Based on the results, conceptual, methodological, and practical implications for training effectiveness and safety culture transformation are discussed.

The Science of Workplace Instruction: Learning and Development Applied to Work

Learning is the engagement in mental processes resulting in the acquisition and retention of knowledge, skills, and/or affect over time and applied when needed. Building on this definition, we integrate the science of training and the science of learning to propose a new science of workplace instruction, linking the design of instructional events to instructional outcomes such as transfer and job performance through the mediating effects of learner events and learning outcomes. We propose three foundational elements: the learner, instructional principles, and training delivery (methods and media). Understanding and applying instructional principles are the primary methods for enhancing training effectiveness; thus, we detail 15 empirically supported principles. We then discuss the erroneous pursuit of aptitude-by-treatment interactions under the guise of learner styles and age-specific instruction. Finally, we offer suggestions for future research that draw on the foundation of instructional principles to optimize self-directed learning and learning in synthetic learning environments.

Investigating voice in action teams: a critical review

Team communication is considered a key factor for team performance. Importantly, voicing concerns and suggestions regarding work-related topics-also termed speaking up-represents an essential part of team communication. Particularly in action teams in high-reliability organizations such as healthcare, military, or aviation, voice is crucial for error prevention. Although research on voice has become more important recently, there are inconsistencies in the literature. This includes methodological issues, such as how voice should be measured in different team contexts, and conceptual issues, such as uncertainty regarding the role of the voice recipient. We tried to address these issues of voice research in action teams in the current literature review. We identified 26 quantitative empirical studies that measured voice as a distinct construct. Results showed that only two-thirds of the articles provided a definition for voice. Voice was assessed via behavioral observation or via self-report. Behavioral observation includes two main approaches (i.e., event-focused and language-focused) that are methodologically consistent. In contrast, studies using self-reports showed significant methodological inconsistencies regarding measurement instruments (i.e., self-constructed single items versus validated scales). The contents of instruments that assessed voice via self-report varied considerably. The recipient of voice was poorly operationalized (i.e., discrepancy between definitions and measurements). In sum, our findings provide a comprehensive overview of how voice is treated in action teams. There seems to be no common understanding of what constitutes voice in action teams, which is associated with several conceptual as well as methodological issues. This suggests that a stronger consensus is needed to improve validity and comparability of research findings.

Applying safety lessons from aviation to pre-licensure health professions education: A narrative critical review

BACKGROUND

Significant numbers of patients continue to be harmed annually by healthcare systems in the United States (US) and around the world. Through a lens of safety, the fields of aviation and healthcare share many similarities in the non-technical skills required by team members, including situational awareness, communication, problem-solving, and leadership. Despite these links and evidence of effective interventions in the clinical setting, there is a lack of a guidance on how to incorporate non-technical skills training into pre-licensure health professions curricula.

METHODS

Following guidance for a narrative critical review, a comprehensive literature search was conducted looking for studies incorporating non-technical skills training including crew resource management (CRM) and human factors and ergonomics (HFE) into pre-licensure health professions curricula.

RESULTS

Eleven example articles were organized into three broad themes: (1) changing the teaching paradigm around errors, (2) targeted curricular interventions, and (3) interprofessional team training. Several useful tools for evaluating training effectiveness were highlighted, but consistent measures of efficacy for CRM/HFE training are lacking. Interprofessional team training may have the most tangible and broadly applicable link to pre-licensure curricula.

IMPLICATIONS

Additional research is needed to identify best practices for consistent incorporation of non-technical skills into pre-licensure curricula. A cultural shift to focus on error management (vs. solely error avoidance) is also needed early in training with development of a common language to discuss patient safety issues and opportunities for improvement across various healthcare settings.

Teamwork and Safety Attitudes in Complex Aortic Surgery at a Dutch Hospital: Cross-Sectional Survey Study

BACKGROUND

Improving teamwork in surgery is a complex goal and difficult to achieve. Human factors questionnaires, such as the Safety Attitudes Questionnaire (SAQ), can help us understand medical teamwork and may assist in achieving this goal.

OBJECTIVE

This paper aimed to assess local team and safety culture in a cardiovascular surgery setting to understand how purposeful teamwork improvements can be reached.

METHODS

Two cardiovascular surgical teams performing complex aortic treatments were assessed: an endovascular-treatment team (ETT) and an open-treatment team (OTT). Both teams answered an online version of the SAQ Dutch Edition (SAQ-NL) consisting of 30 questions related to six different domains of safety: teamwork climate, safety climate, job satisfaction, stress recognition, perceptions of management, and working conditions. In addition, one open-ended question was posed to gain more insight into the completed questionnaires.

RESULTS

The SAQ-NL was completed by all 23 ETT members and all 13 OTT members. Team composition was comparable for both teams: 57% and 62% males, respectively, and 48% and 54% physicians, respectively. All participants worked for 10 years or more in health care. SAQ-NL mean scores were comparable between both teams, with important differences found between the physicians and nonphysicians of the ETT. Nonphysicians were less positive about the safety climate, job satisfaction, and working climate domains than were the physicians (P<.05). Additional education on performed procedures, more conjoined team training, as well as a hybrid operating room were suggested by participants as important areas of improvement.

CONCLUSIONS

Nonphysicians of a local team performing complex endovascular aortic aneurysm surgery perceived safety climate, job satisfaction, and working conditions less positively than did physicians from the same team. Open-ended questions suggested that this is related to a lack of adequate conjoined training, lack of adequate education, and lack of an adequate operating room. With added open-ended questions, the SAQ-NL appears to be an assessment tool that allows for developing strategies that are instrumental in improving quality of care.

How to communicate between surgeon and intensivist?

PURPOSE OF REVIEW

Communication and teamwork are essential to enhance the quality of care, especially in operating rooms and ICUs. In these settings, the effective interprofessional collaboration between surgeons and intensivists impacts patients' outcome. This review discusses current opinions and evidence for improving communication strategies and the relationship between surgeons and intensivists/anesthesiologist.

RECENT FINDINGS

Effective teamwork has been demonstrated to improve patient outcome and foster healthier relationships between professionals.With the expansion of new medical superspecialist disciplines and the latest medical developments, patient care has been put through a progressive fragmentation, rather than a holistic approach. Operating theaters and ICU are the common fields where surgeons and anesthesiologists/intensivists work. However, communication challenges may frequently arise. Therefore, effective communication, relational coordination, and team situation awareness are considered to affect quality of teamwork in three different phases of the patient-centered care process: preoperatively, intraoperatively, and postoperatively.

SUMMARY

Although limited, current evidence suggests to improve communication and teamwork in patient perioperative care. Further research is needed to strengthen the surgeon-intensivist relationship and to deliver high-quality patient care.

Anaesthesiology students' Non-Technical skills: development and evaluation of a behavioural marker system for students (AS-NTS)

BACKGROUND

Non-Technical Skills (NTS) are becoming more important in medical education. A lack of NTS was identified as a major reason for unsafe patient care, favouring adverse events and team breakdown. Therefore, the training of NTS should already be implemented in undergraduate teaching. The goal of our study was to develop and validate the Anaesthesiology Students' Non-Technical Skills (AS-NTS) as a feasible rating tool to assess students' NTS in emergency and anaesthesiology education.

METHODS

The development of AS-NTS was empirically grounded in expert- and focus groups, field observations and data from NTS in medical fields. Validation, reliability and usability testing was conducted in 98 simulation scenarios, during emergency and anaesthesiology training sessions.

RESULTS

AS-NTS showed an excellent interrater reliability (mean 0.89), achieved excellent content validity indexes (at least 0.8) and was rated as feasible and applicable by educators. Additionally, we could rule out the influence of the raters' anaesthesiology and emergency training and experience in education on the application of the rating tool.

CONCLUSIONS

AS-NTS provides a structured approach to the assessment of NTS in undergraduates, providing accurate feedback. The findings of usability, validity and reliability indicate that AS-NTS can be used by anaesthesiologists in different year of postgraduate training, even with little experience in medical education.

The training and practice of crew resource management: recommendations from an inductive in vivo study of the flight deck

Crew resource management (CRM) is credited with saving 185 lives at Sioux City. While the theory behind CRM is well documented, there are few studies of how CRM manifests on the line. This inductive in vivo study had three objectives. First, to describe how CRM manifests. Secondly, to evaluate the efficacy of CRM vis-à-vis flight safety. Thirdly, to suggest improvements to the CRM training syllabus. The study produced five conclusions: First, CRM is durable under conditions of moderate strain. Secondly, crews embed and refine CRM through reflection and action. Thirdly, CRM facilitates and shapes social relations. Fourthly, mindlessness (Langer 1989 ) undermines CRM. Finally, the interruption of flight-deck routines by third-parties poses a threat to flight-safety. The paper recommends multi-profession CRM training as a means of improving communication and co-ordination in and around aircraft. The study's limitations include a monocultural flight-deck: flights were operated by pilots with European backgrounds. Mindful of Hofstede's ( 1980 ), Engle's ( 2000 ) and Helmreich and Merritt's ( 2001 ) examination of the relationship between culture and performance, the author suggests the study be repeated with carriers that employ pilots from a variety of cultures. Practitioner Summary: This in vivo study evaluates the efficacy of CRM vis-à-vis flight safety and supports a critique of the CRM syllabus. The author observed twenty sectors and attended a CRM training day. To improve safety and efficiency, it is recommended that airlines run multi-profession (inclusive) CRM training courses.

A comparative study on the frequency of simulation-based training and assessment of non-technical skills in the Norwegian ground ambulance services and helicopter emergency medical services

BACKGROUND

Inadequate non-technical skills (NTSs) among employees in the Norwegian prehospital emergency medical services (EMSs) are a risk for patient and operational safety. Simulation-based training and assessment is promising with respect to improving NTSs. The frequency of simulation-based training in and assessment of NTSs among crewmembers in the Norwegian helicopter emergency medical service (HEMS) has gained increased attention over recent years, whereas there has been much less focus on the Norwegian ground emergency medical service (GEMS). The aim of the study was to compare and document the frequencies of simulation-based training in and assessment of seven NTSs between the Norwegian HEMS and GEMS, conditional on workplace and occupation.

METHOD

A comparative study of the results from cross-sectional questionnaires responded to by employees in the Norwegian prehospital EMSs in 2016 regarding training in and assessment of NTSs during 2015, with a focus on the Norwegian GEMS and HEMS. Professional groups of interest are: pilots, HEMS crew members (HCMs), physicians, paramedics, emergency medical technicians (EMTs), EMT apprentices, nurses and nurses with an EMT licence.

RESULTS

The frequency of simulation-based training in and assessment of seven generic NTSs was statistically significantly greater for HEMS than for GEMS during 2015. Compared with pilots and HCMs, other health care providers in GEMS and HEMS undergo statistically significantly less frequent simulation-based training in and assessment of NTSs. Physicians working in the HEMS appear to be undergoing training and assessment more frequently than the rest of the health trust employees. The study indicates a tendency for lesser focus on the assessment of NTSs compared to simulation-based training.

CONCLUSION

HEMS has become superior to GEMS, in terms of frequency of training in and assessment of NTSs. The low frequency of training in and assessment of NTSs in GEMS suggests that there is a great potential to learn from HEMS and to strengthen the focus on NTSs. Increased frequency of assessment of NTSs in both HEMS and GEMS is called for.

Transfer of Training: The Known and the Unknown

Transfer of training is one of the oldest topics of interest to industrial and organizational (I/O) psychologists. Drawing on several meta-analytic studies and recent empirical work, we first synthesize what is now reliably known with respect to the generalization and retention of learned knowledge and skills to work contexts. The second part of our review focuses on what is unknown—the significant gaps in our knowledge where we believe new directions in our research strategies are warranted. We offer three prescriptions: (a) going one step beyond most existing studies to offer greater precision in our specification and measurement of variables and interventions, (b) connecting the dots by focusing on transfer criteria and transfer trajectories, and (c) shifting the operative paradigm of research to examine contemporary learning from a problem-centered perspective. There is ample opportunity to increase the yield on enormous organizational investments in training if transfer scholars and practitioners are fully informed of what is known and prepared to systematically confront the unknown in new and innovative ways.

Teambuilding-Maßnahmen für Erzieherinnen und Erzieher in Kindertagesstätten

Zusammenfassung. Teambuilding-Maßnahmen im betrieblichen Setting thematisieren in der Regel Aspekte wie Zielsetzung, Beziehungs- und Rollenklärung oder Problemlösung und bieten die Möglichkeit, außerhalb des alltäglichen Berufslebens Themen und Sachverhalte mit anderen Teammitgliedern zu besprechen und zu bearbeiten. Gleichzeitig beinhalten solche Maßnahmen das Potential zur individuellen und teambezogenen Weiterentwicklung, die sich in einer Stärkung des Teamzusammenhalts oder einer verbesserten Kommunikation im Team äußern kann. Die Wirksamkeit von derartigen Personalentwicklungsmaßnahmen konnte bereits in einer Reihe von Studien bestätigt werden. Die vorliegende Untersuchung soll zwei zentrale Fragen klären. In einem Prä-Post-Follow-up Design soll zum einen überprüft werden, welchen Einfluss eine einmalige Teambuilding-Maßnahme auf das Teamklima von Kita-Erziehern (N = 39) hat. Zum anderen soll überprüft werden, inwieweit die Wahl einer passiven bzw. aktiven Vermittlungsmethode diesen Effekt beeinflusst. Die Ergebnisse zeigen, dass eine aktive Vermittlungsmethode keinen Vorteil gegenüber einer passiven Methode hat.

Does teaching non-technical skills to medical students improve those skills and simulated patient outcome?

OBJECTIVES

The purpose of this study is to evaluate the effects of a tailor-made, non-technical skills seminar on medical student's behaviour, attitudes, and performance during simulated patient treatment.

METHODS

Seventy-seven students were randomized to either a non-technical skills seminar (NTS group, n=43) or a medical seminar (control group, n=34). The human patient simulation was used as an evaluation tool. Before the seminars, all students performed the same simulated emergency scenario to provide baseline measurements. After the seminars, all students were exposed to a second scenario, and behavioural markers for evaluating their non-technical skills were rated. Furthermore, teamwork-relevant attitudes were measured before and after the scenarios, and perceived stress was measured following each simulation. All simulations were also evaluated for various medical endpoints.

RESULTS

Non-technical skills concerning situation awareness (p<.01, r=0.5) and teamwork (p<.01, r=0.45) improved from simulation I to II in the NTS group. Decision making improved in both groups (NTS: p<.01, r=0.39; control: p<.01, r=0.46). The attitude 'handling errors' improved significantly in the NTS group (p<.05, r=0.34). Perceived stress decreased from simulation I to II in both groups. Medical endpoints and patients´ outcome did not differ significantly between the groups in simulation II.

CONCLUSIONS

This study highlights the effectiveness of a single brief seminar on non-technical skills to improve student's non-technical skills. In a next step, to improve student's handling of emergencies and patient outcomes, non-technical skills seminars should be accompanied by exercises and more broadly embedded in the medical school curriculum.

The Automation-by-Expertise-by-Training Interaction

OBJECTIVE

I introduce the automation-by-expertise-by-training interaction in automated systems and discuss its influence on operator performance.

BACKGROUND

Transportation accidents that, across a 30-year interval demonstrated identical automation-related operator errors, suggest a need to reexamine traditional views of automation.

METHOD

I review accident investigation reports, regulator studies, and literature on human computer interaction, expertise, and training and discuss how failing to attend to the interaction of automation, expertise level, and training has enabled operators to commit identical automation-related errors.

RESULTS

Automated systems continue to provide capabilities exceeding operators' need for effective system operation and provide interfaces that can hinder, rather than enhance, operator automation-related situation awareness. Because of limitations in time and resources, training programs do not provide operators the expertise needed to effectively operate these automated systems, requiring them to obtain the expertise ad hoc during system operations. As a result, many do not acquire necessary automation-related system expertise.

CONCLUSION

Integrating automation with expected operator expertise levels, and within training programs that provide operators the necessary automation expertise, can reduce opportunities for automation-related operator errors.

APPLICATION

Research to address the automation-by-expertise-by-training interaction is needed. However, such research must meet challenges inherent to examining realistic sociotechnical system automation features with representative samples of operators, perhaps by using observational and ethnographic research. Research in this domain should improve the integration of design and training and, it is hoped, enhance operator performance.

Team Training for Dynamic Cross-Functional Teams in Aviation: Behavioral, Cognitive, and Performance Outcomes

OBJECTIVE

This study evaluates the effectiveness of a training program designed to improve cross-functional coordination in airline operations.

BACKGROUND

Teamwork across professional specializations is essential for safe and efficient airline operations, but aviation education primarily emphasizes positional knowledge and skill. Although crew resource management training is commonly used to provide some degree of teamwork training, it is generally focused on specific specializations, and little training is provided in coordination across specializations.

METHOD

The current study describes and evaluates a multifaceted training program designed to enhance teamwork and team performance of cross-functional teams within a simulated airline flight operations center. The training included a variety of components: orientation training, position-specific declarative knowledge training, position-specific procedural knowledge training, a series of high-fidelity team simulations, and a series of after-action reviews.

RESULTS

Following training, participants demonstrated more effective teamwork, development of transactive memory, and more effective team performance.

CONCLUSION

Multifaceted team training that incorporates positional training and team interaction in complex realistic situations and followed by after-action reviews can facilitate teamwork and team performance.

APPLICATION

Team training programs, such as the one described here, have potential to improve the training of aviation professionals. These techniques can be applied to other contexts where multidisciplinary teams and multiteam systems work to perform highly interdependent activities.

A meta-analysis of the effectiveness of crew resource management training in acute care domains

The healthcare industry has seen an increase in the adoption of team training, such as crew resource management (CRM), to improve teamwork and coordination within acute care medical teams. A meta-analysis was carried out in order to quantify the effects of CRM training on reactions, learning, behaviour and clinical care outcomes. Biases in the research evidence are identified and recommendations for training development and evaluation are presented. PUBMED, EMBASE and PsychInfo were systematically searched for all relevant papers. Peer reviewed papers published in English between January 1985 and September 2013, which present empirically based studies focusing on interventions to improve team effectiveness in acute health care domains, were included. A total of 20 CRM-type team training evaluation studies were found to fulfil the a priori criteria for inclusion in the meta-analysis. Overall, CRM trained participants responded positively to CRM (mean score 4.25 out of a maximum of 5), the training had large effects on participants' knowledge (d=1.05), a small effect on attitudes (d=0.22) and a large effect on behaviours (d=1.25). There was insufficient evidence to support an effect on clinical care outcomes or long term impacts. The findings support the premise that CRM training can positively impact teamwork in healthcare and provide estimates of the expected effects of training. However, there is a need for greater precision in outcome assessment, improved standardisation of methods and measures, and more robust research design. Stronger evidence of effectiveness will require multi-level, multicentre, multispecialty and longitudinal studies.

Simulation-based training and assessment of non-technical skills in the Norwegian Helicopter Emergency Medical Services: a cross-sectional survey

BACKGROUND

Human error and deficient non-technical skills (NTSs) among providers of ALS in helicopter emergency medical services (HEMS) is a threat to patient and operational safety. Skills can be improved through simulation-based training and assessment.

OBJECTIVE

To document the current level of simulation-based training and assessment of seven generic NTSs in crew members in the Norwegian HEMS.

METHODS

A cross-sectional survey, either electronic or paper-based, of all 207 physicians, HEMS crew members (HCMs) and pilots working in the civilian Norwegian HEMS (11 bases), between 8 May and 25 July 2012.

RESULTS

The response rate was 82% (n=193). A large proportion of each of the professional groups lacked simulation-based training and assessment of their NTSs. Compared with pilots and HCMs, physicians undergo statistically significantly less frequent simulation-based training and assessment of their NTSs. Fifty out of 82 (61%) physicians were on call for more than 72 consecutive hours on a regular basis. Of these, 79% did not have any training in coping with fatigue. In contrast, 72 out of 73 (99%) pilots and HCMs were on call for more than 3 days in a row. Of these, 54% did not have any training in coping with fatigue.

CONCLUSIONS

Our study indicates a lack of simulation-based training and assessment. Pilots and HCMs train and are assessed more frequently than physicians. All professional groups are on call for extended hours, but receive limited training in how to cope with fatigue.

Utility and assessment of non-technical skills for rapid response systems and medical emergency teams

Efforts are ongoing to improve outcomes from cardiac arrest and medical emergencies. A promising quality improvement modality is use of non-technical skills (NTS) that aim to address human factors through improvements in performance of leadership, communication, situational awareness and decision-making. Originating in the airline industry, NTS training has been successfully introduced into anaesthesia, surgery, emergency medicine and other acute medical specialities. Some aspects of NTS have already achieved acceptance for cardiac arrest teams. Leadership skills are emphasised in advanced life support training and have shown favourable results when employed in simulated and clinical resuscitation scenarios. The application of NTS in medical emergency teams as part of a rapid response system attending medical emergencies is less certain; however, observations of simulations have also shown promise. This review highlights the potential benefits of NTS competency for cardiac arrest teams and, more importantly, medical emergency teams because of the diversity of clinical scenarios encountered. Discussion covers methods to assess and refine NTS and NTS training to optimise performance in the clinical environment. Increasing attention should be applied to yielding meaningful patient and organisational outcomes from use of NTS. Similarly, implementation of any training course should receive appropriate scrutiny to refine team and institutional performance.

The effects of Crew Resource Management (CRM) training on flight attendants' safety attitudes

INTRODUCTION

A number of well-known incidents and accidents had led the aviation industry to introduce Crew Resource Management (CRM) training designed specifically for flight attendants, and joint (pilot and flight attendant) CRM training as a way to improve teamwork and communication. The development of these new CRM training programs during the 1990s highlighted the growing need for programs to be evaluated using research tools that had been validated for the flight attendant population.

METHOD

The FSAQ (Flight Safety Attitudes Questionnaire-Flight Attendants) was designed specifically to obtain safety attitude data from flight attendants working for an Asia-Pacific airline. Flight attendants volunteered to participate in a study before receiving CRM training (N=563) and again (N=526) after CRM training.

RESULTS

Almost half (13) of the items from the 36-item FSAQ showed highly significant changes following CRM training. Years of experience, crew position, seniority, leadership roles, flight attendant crew size, and length of route flown were all predictive of safety attitudes.

PRACTICAL APPLICATIONS

CRM training for flight attendants is a valuable tool for increasing positive teamwork behaviors between the flight attendant and pilot sub-groups. Joint training sessions, where flight attendants and pilots work together to find solutions to in-flight emergency scenarios, provide a particularly useful strategy in breaking down communication barriers between the two sub-groups.

On a wing and a prayer: an assessment of modularized crew resource management training for health care professionals

INTRODUCTION

Evidence suggests that Crew Resource Management (CRM), a form of team training, is beneficial. In CRM training, participants learn individual portable team skills such as communication and decision making through group discussion and activities. However, the usual 1-day course format is not always compatible with health care organizational routines. A modular training format, while theoretically sound, is untested for interprofessional team training. The aim of this study was to explore the potential for modularized CRM training to be delivered to a group of interprofessional learners.

METHOD

Modularized CRM training, consisting of two 2-hour workshops, was delivered to health care workers in an Australian tertiary hospital. Kirkpatrick's evaluation model provided a framework for the study. Baseline attitude surveys were conducted prior to each workshop. Participants completed a written questionnaire at the end of each workshop that examined their motivations, reactions to the training, and learner demographics. An additional survey, administered 6 weeks post training, captured self-assessed behavior data.

RESULTS

Twenty-three individuals from a range of professions and clinical streams participated. One in 5 participants (22%) reported that they translated teamwork skills to the workplace. While positive about the workshop format and content, many respondents identified personal, team, and organizational barriers to the application of the workshop techniques.

DISCUSSION

CRM training when delivered in a modular format has positive outcomes. Following the training, some respondents overcame workplace barriers to attempt to change negative workplace behavior. This progress provides cautious optimism for the potential for modular CRM training to benefit groups of interprofessional health staff.

"Excuse me:" teaching interns to speak up

BACKGROUND

A consistent finding in health care is that interns (junior physicians) are unwilling to "speak up" to attendings or consultants (senior physicians), which involves the related behaviors of challenging or questioning the actions of attendings, seeking guidance about one's own performance, and reporting one's errors. Training designed to encourage interns to speak up was designed, conducted, and evaluated at University Hospital Galway, an academic teaching hospital affiliated with the National University of Ireland, Galway, in January 2012-March 2012 and November 2012-January 2013.

METHOD

Training, which was based on the Crew Resource Management model, was constructed around filmed stories of attending physicians describing situations in which, when they were interns, their communication and assertiveness skills were challenged, and their reflections on what they could have done better.

RESULTS

A total of 110 interns attended the training. The feedback from participants was positive. There was a significant increase in knowledge as a result of the training, and some evidence to support a shift in attitudes in the desirable direction relating to the need to speak up to seniors. No effect of the training was found on behavior.

CONCLUSIONS

The willingness to share examples of poor performance is an important part of building a good safety culture. Listening to the stories ofattendings describing and reflecting on a situation in which they struggled is a powerful teaching method for training interns in the communication and assertiveness skills that are critical to their job. However, a sustained change in attitudes and behavior to speaking up will require a reinforcement of learning with deliberate practice to develop the skills introduced through the training program.

Is Crew Resource Management Training Effective in Healthcare Settings?

The healthcare industry has seen a recent proliferation in the adoption of team training, such as crew resource management (CRM). Empirical studies of the effectiveness of CRM-type training in healthcare were subjected to meta-analysis. Fourteen CRM evaluation studies were found to fulfill the a priori criteria for inclusion in the meta-analysis. The metrics of CRM training effectiveness analyzed were: reactions, attitudes, knowledge, behaviors, and organization. CRM trained participants responded positively to CRM, the training had large effects on the participants’ knowledge and behaviors, and a medium effect on their attitudes. The training was also found to have a medium effect on the number of errors/adverse events and a small effect on the time to complete clinical tasks. Although there are methodological limitations that mean the results from this study should be treated with a little caution, the findings are nevertheless encouraging.

Nursing crew resource management: a follow-up report from the Veterans Health Administration

In response to low scores on a patient safety culture survey, the Veterans Health Administration National Center for Patient Safety implemented a comprehensive nursing-focused crew resource management program for frontline nursing staff. This article highlights significant cultural and clinical outcomes from the program.

Developing a Single-Pilot Line Operations Safety Audit

A single-pilot form of the line operations safety audit was trialed with a mid-sized emergency medical service air operator using two observers with a sample of pilots flying 14 sectors. The conceptual basis for observing pilot performance and analyzing data was the threat and error management model, focusing on threats, errors, undesired aircraft states, and their management. Forty-six threats and 42 crew errors were observed. Pilots generally used sound strategies to prevent errors and to manage successfully those that occurred. Threats resulting from operational pressures were well managed. The study achieved its objective of determining whether a single-pilot line operations safety audit could be successfully developed and used as a basis for systematic data collection.

Teamwork and team training in the ICU: where do the similarities with aviation end?

The aviation industry has made significant progress in identifying the skills and behaviors that result in effective teamwork. Its conceptualization of teamwork, development of training programs, and design of assessment tools are highly relevant to the intensive care unit (ICU). Team skills are important for maintaining safety in both domains, as multidisciplinary teams must work effectively under highly complex, stressful, and uncertain conditions. However, there are substantial differences in the nature of work and structure of teams in the ICU in comparison with those in aviation. While intensive care medicine may wish to use the advances made by the aviation industry for conceptualizing team skills and implementing team training programs, interventions must be tailored to the highly specific demands of the ICU.

A comparison of the teamwork attitudes and knowledge of Irish surgeons and U.S Naval aviators

Poor teamwork skills are contributors to poor performance and mishaps in high risk work settings, including the operating theatre. A questionnaire was used to assess the attitudes towards, and knowledge of, Irish surgeons (n = 72) towards the human factors that contribute to mishaps and poor teamwork in high risk environments. The responses were compared to those obtained from U.S. Naval aviators (n = 552 for the attitude questions, and n = 172 for the knowledge test). U.S. Naval aviators were found to be significantly more knowledgeable, and held attitudes that were significantly more positive towards effective teamworking than the surgeons. Moreover, 78.9% of Senior House Officers and Registrars stated that junior personnel were frequently afraid to speak-up (compared with 31.3% of Consultants). Only 7.3% of surgeons stated that an adequate pre-operative brief team brief was frequently conducted, and only 15% stated that an adequate post-operative team brief was frequently conducted. It is suggested that the human factors training currently provided to surgeons in Ireland is a positive first step. However, there is a need to stress the importance of assertiveness in juniors, listening in seniors, and more reinforcement of good teamworking behaviours in the operating theatre.

The anatomy of health care team training and the state of practice: a critical review

PURPOSE

As the U.S. health care system enters a new era, the importance of team-based care approaches grows. How is the health care community ensuring that providers and administrators are equipped with the knowledge, skills, and attitudes (KSAs) foundational for effective teamwork? Are these KSAs transferring into daily practice? This review summarizes the present state of practice for health care team training described in published literature. Drawing from empirical investigations of training effectiveness, the authors explore training design, implementation, and evaluation to provide insight into the shape, structure, and anatomy of team training in health care.

METHOD

A 2009 literature search yielded 40 peer-reviewed articles detailing health care team training evaluations. Guided by 11 focal questions, two trained raters extracted details regarding training design, implementation, evaluation metrics, and outcomes.

RESULTS

Findings indicate that team training is being implemented across a wide spectrum of providers and is primarily targeting communication, situational awareness, leadership, and role clarity. Relatively few details indicate how training needs were established. Most studies collected data immediately posttraining; however, less than 30% collected data six months or more posttraining. Content analyses highlight the need for enhanced detail in published training evaluation reports.

CONCLUSIONS

In many respects, health care team training implementation and evaluation align with best practices suggested from the science of training, adult learning, and human performance; however, opportunities for improvement exist. The authors suggest several mechanisms for furthering the health care team training evidence base to enhance patient safety and work environment quality for clinicians.

Crew resource management training within the automotive industry: does it work?

OBJECTIVE

This article presents the development, implementation, and evaluation of a crew resource management (CRM) training program specifically designed for employees within the automotive industry.

BACKGROUND

The central objective of this training program was to improve communication, teamwork, and stress management skills as well to increase the workers' situational awareness of potential errors that can occur during the production process.

METHOD

Participants in the training program of this study were 80 employees, all of whom were working in a production unit for gearbox manufacturing. Effectiveness of the CRM training course was evaluated two times (1 month and 6 months after the training program).

RESULTS

The results showed a significant improvement in a wide range of CRM-relevant categories, especially in teamwork-related attitudes, in addition to an increase in the workers' situational awareness after the training program.

CONCLUSION

On the basis of the results, it can be stated that CRM training, which was originally developed for the aviation industry, can be transferred to the automotive industry. However, because of the lack of behavioral observations, these effects are limited to CRM attitudes and knowledge changes.

APPLICATION

Several recommendations for future research and training development in the field of human factors training are made.

Adaptive coordination and heedfulness make better cockpit crews

Team coordination during a simulated clean approach performed by 42 civil aviation cockpit crews was analysed. Several hypotheses regarding the adaptive use of implicit and explicit coordination, leadership and heedful interrelating were tested. The results indicate the adaptiveness of coordination to different levels of standardisation and task load and the general importance of explicit coordination for good performance. Leadership seems to be required mainly for work phases with little standardisation. In exploratory lag sequential analyses, heedful behaviour in the seven best and six worst performing crews was examined. The coordination sequences in high performance crews were found to be more succinct and well balanced, indicating that a shared sense of heedfulness is crucial for effectiveness. Theoretical implications for the conceptualisation of adaptive coordination and heedfulness and practical implications for improving crew training are discussed. Statement of Relevance: Analyses of team coordination during a simulated clean approach performed by civil aviation cockpit crews demonstrated the occurrence and effectiveness of adaptive coordination in response to different levels of task load and standardisation. Results also indicated the importance of heedful interrelating, both as a form of coordination and as a way of regulating the adaptiveness of coordination efforts. These findings have important implications for improving crew training, leadership practices and possibly also standard operating procedures.

Integrating the science of team training: guidelines for continuing education

The provision of high-quality, efficient care results from the coordinated, cooperative efforts of multiple technically competent health care providers working in concert over time, spanning disciplinary and professional boundaries. Accordingly, the role of medical education must include the development of providers who are both expert clinicians and expert team members. However, the competencies underlying effective teamwork are only just beginning to be integrated into medical school curricula and residency programs. Therefore, continuing education (CE) is a vital mechanism for practitioners already in the field to develop the attitudes, behaviors (skills), and cognitive knowledge necessary for highly reliable and effective team performance.The present article provides an overview of more than 30 years of evidence regarding team performance and team training in order to guide, shape, and build CE activities that focus on developing team competencies. Recognizing that even the most comprehensive and well-designed team-oriented CE programs will fail unless they are supported by an organizational and professional culture that values collaborative behavior, ten evidence-based lessons for practice are offered in order to facilitate the use of the science of team-training in efforts to foster continuous quality improvement and enhance patient safety.